{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Converting Data for Visualization\n",
    "\n",
    "Although we've managed to extract a few examples of both dabs and tposes, it's now time to figure out what our data looks like. \n",
    "\n",
    "The easiest way to manipulate and visualize data in Python is via tools like Pandas and Seaborn. \n",
    "\n",
    "But first, we'll need to convert our numpy raw arrays into something that's a bit more readable. So let's do that by converting them into labeled CSV files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "np.random.seed(1337)\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "dabDataset = np.load('data/dabs.npy')\n",
    "tposeDataset = np.load('data/tposes.npy')\n",
    "otherDataset = np.load('data/other.npy')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[5.8832416e+02, 2.9433704e+02, 7.2265184e-01],\n",
       "       [5.8239331e+02, 3.5126093e+02, 8.0205584e-01],\n",
       "       [5.0984329e+02, 3.4919385e+02, 7.5316119e-01],\n",
       "       [4.1784265e+02, 3.1985785e+02, 8.1164622e-01],\n",
       "       [3.6101605e+02, 2.9243521e+02, 8.0296052e-01],\n",
       "       [6.5091376e+02, 3.6097537e+02, 6.4161348e-01],\n",
       "       [6.3724268e+02, 2.7274924e+02, 7.8188539e-01],\n",
       "       [4.9614203e+02, 2.4154723e+02, 8.3243752e-01],\n",
       "       [5.4315808e+02, 6.4114813e+02, 4.4807938e-01],\n",
       "       [4.8636816e+02, 6.2938318e+02, 3.6906898e-01],\n",
       "       [0.0000000e+00, 0.0000000e+00, 0.0000000e+00],\n",
       "       [0.0000000e+00, 0.0000000e+00, 0.0000000e+00],\n",
       "       [6.0191382e+02, 6.4702966e+02, 3.8946095e-01],\n",
       "       [0.0000000e+00, 0.0000000e+00, 0.0000000e+00],\n",
       "       [0.0000000e+00, 0.0000000e+00, 0.0000000e+00],\n",
       "       [5.7648334e+02, 2.7475522e+02, 6.1822432e-01],\n",
       "       [6.0389270e+02, 2.8454663e+02, 4.1854110e-01],\n",
       "       [5.5686536e+02, 2.6891223e+02, 2.7014270e-01],\n",
       "       [6.1959991e+02, 2.9243130e+02, 7.0310913e-02],\n",
       "       [0.0000000e+00, 0.0000000e+00, 0.0000000e+00],\n",
       "       [0.0000000e+00, 0.0000000e+00, 0.0000000e+00],\n",
       "       [0.0000000e+00, 0.0000000e+00, 0.0000000e+00],\n",
       "       [0.0000000e+00, 0.0000000e+00, 0.0000000e+00],\n",
       "       [0.0000000e+00, 0.0000000e+00, 0.0000000e+00],\n",
       "       [0.0000000e+00, 0.0000000e+00, 0.0000000e+00]], dtype=float32)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dabDataset[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(25, 3)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dabDataset[0].shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Adding our Labels\n",
    "\n",
    "Our labels come from the [BODY_25 Pose Output format](https://github.com/CMU-Perceptual-Computing-Lab/openpose/blob/master/doc/output.md#pose-output-format-body_25) available at the repo. \n",
    "\n",
    "We can tell because when we looked at each of our poses, we saw a `dataset[0].shape` of 25. This matches the number of labels below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "labels = [\"Nose\", \"Neck\", \"RShoulder\", \"RElbow\", \"RWrist\", \"LShoulder\", \"LElbow\",\n",
    " \"LWrist\", \"MidHip\", \"RHip\", \"RKnee\", \"RAnkle\", \"LHip\", \"LKnee\", \"LAnkle\",\n",
    " \"REye\", \"LEye\", \"REar\", \"LEar\", \"LBigToe\", \"LSmallToe\", \"LHeel\", \"RBigToe\",\n",
    " \"RSmallToe\", \"RHeel\", \"Background\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each of our labels comes as an `X`, `Y`, and `Confidence`. Let's add those labels and flatten this array for our CSV file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "properLabels = []\n",
    "for label in labels:\n",
    "    properLabels.append(label + 'X')\n",
    "    properLabels.append(label + 'Y')\n",
    "    properLabels.append(label + 'Confidence')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "import csv\n",
    "\n",
    "with open('data/dabs.csv', 'w+') as dabcsv:\n",
    "    dabwriter = csv.writer(dabcsv, delimiter=',')\n",
    "    dabwriter.writerow(properLabels)\n",
    "    for cell in dabDataset:\n",
    "        dabwriter.writerow(cell.flatten())\n",
    "        \n",
    "with open('data/tposes.csv', 'w+') as tposecsv:\n",
    "    tposewriter = csv.writer(tposecsv, delimiter=',')\n",
    "    tposewriter.writerow(properLabels)\n",
    "    for cell in tposeDataset:\n",
    "        tposewriter.writerow(cell.flatten())\n",
    "        \n",
    "with open('data/other.csv', 'w+') as othercsv:\n",
    "    otherwriter = csv.writer(othercsv, delimiter=',')\n",
    "    otherwriter.writerow(properLabels)\n",
    "    for cell in otherDataset:\n",
    "        otherwriter.writerow(cell.flatten())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sanity Checking our Data\n",
    "\n",
    "We can now open up our CSV files and see what they look like. How many samples did we collect? Is it enough? \n",
    "\n",
    "Once we check, we can hop on to the next step, bringing all the data into a single format and file for training."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating a Labeled Dataset for Training and Testing\n",
    "\n",
    "Now that we've got our data (mostly) sorted out, we need to convert it into a set. \n",
    "\n",
    "We'll use `0` for `other` poses, `1` for `dabs`, and `2` for `tposes`.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
      " 0. 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 2. 2. 2. 2. 2. 2. 2. 2.\n",
      " 2. 2. 2. 2. 2. 2. 2. 2.]\n",
      "56 total examples for training.\n"
     ]
    }
   ],
   "source": [
    "labels = np.zeros(len(otherDataset))\n",
    "labels = np.append(labels, np.full((len(dabDataset)), 1))\n",
    "labels = np.append(labels, np.full((len(tposeDataset)), 2))\n",
    "print(labels)\n",
    "print(\"%i total examples for training.\" % len(labels))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[[488.3213     147.51425      0.83340967]\n",
      "  [494.22372    284.5734       0.8012297 ]\n",
      "  [386.4863     270.83716      0.66853976]\n",
      "  ...\n",
      "  [  0.           0.           0.        ]\n",
      "  [  0.           0.           0.        ]\n",
      "  [  0.           0.           0.        ]]\n",
      "\n",
      " [[515.7737     112.18818      0.83487195]\n",
      "  [478.48004    274.7029       0.8005627 ]\n",
      "  [368.77948    257.2105       0.6782713 ]\n",
      "  ...\n",
      "  [  0.           0.           0.        ]\n",
      "  [  0.           0.           0.        ]\n",
      "  [  0.           0.           0.        ]]\n",
      "\n",
      " [[547.1316     112.15151      0.79948723]\n",
      "  [464.79065    268.88403      0.73338044]\n",
      "  [360.98135    243.43745      0.62600124]\n",
      "  ...\n",
      "  [  0.           0.           0.        ]\n",
      "  [  0.           0.           0.        ]\n",
      "  [  0.           0.           0.        ]]\n",
      "\n",
      " ...\n",
      "\n",
      " [[509.97504    257.06958      0.892523  ]\n",
      "  [460.9663     351.17117      0.7867987 ]\n",
      "  [372.75305    333.54434      0.6111988 ]\n",
      "  ...\n",
      "  [  0.           0.           0.        ]\n",
      "  [  0.           0.           0.        ]\n",
      "  [  0.           0.           0.        ]]\n",
      "\n",
      " [[554.90063    286.50854      0.88104486]\n",
      "  [496.1236     374.6841       0.7804795 ]\n",
      "  [415.7811     353.13678      0.74092144]\n",
      "  ...\n",
      "  [  0.           0.           0.        ]\n",
      "  [  0.           0.           0.        ]\n",
      "  [  0.           0.           0.        ]]\n",
      "\n",
      " [[570.5084     268.88422      0.855286  ]\n",
      "  [509.95016    370.85114      0.7977039 ]\n",
      "  [431.50925    353.08978      0.7552353 ]\n",
      "  ...\n",
      "  [  0.           0.           0.        ]\n",
      "  [  0.           0.           0.        ]\n",
      "  [  0.           0.           0.        ]]]\n"
     ]
    }
   ],
   "source": [
    "dataset = np.append(otherDataset, dabDataset, axis=0)\n",
    "dataset = np.append(dataset, tposeDataset, axis=0)\n",
    "print(dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(56, 25, 3)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.11524551, 0.22232297, 0.21159153, ..., 0.        , 0.        ,\n",
       "        0.        ],\n",
       "       [0.08764701, 0.21461165, 0.2009457 , ..., 0.        , 0.        ,\n",
       "        0.        ],\n",
       "       [0.08761837, 0.21006565, 0.19018552, ..., 0.        , 0.        ,\n",
       "        0.        ],\n",
       "       ...,\n",
       "       [0.20083562, 0.2743525 , 0.26058152, ..., 0.        , 0.        ,\n",
       "        0.        ],\n",
       "       [0.2238348 , 0.29272196, 0.27588812, ..., 0.        , 0.        ,\n",
       "        0.        ],\n",
       "       [0.2100658 , 0.28972745, 0.2758514 , ..., 0.        , 0.        ,\n",
       "        0.        ]], dtype=float32)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset[:,:,1] / 1280"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3\n"
     ]
    }
   ],
   "source": [
    "# now, let's shuffle labels and the array, the same way\n",
    "from sklearn.utils import shuffle\n",
    "X, y = shuffle(dataset, labels)\n",
    "# now let's label them for 'one hot'\n",
    "from keras.utils.np_utils import to_categorical\n",
    "y = to_categorical(y, 3)\n",
    "print(y.shape[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Dropout, Activation, Flatten\n",
    "from keras.optimizers import SGD"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[4.88321289e+02, 1.47514252e+02, 8.33409667e-01],\n",
       "       [4.94223724e+02, 2.84573395e+02, 8.01229715e-01],\n",
       "       [3.86486298e+02, 2.70837158e+02, 6.68539762e-01],\n",
       "       [3.37498718e+02, 4.31440033e+02, 8.06459844e-01],\n",
       "       [2.76727325e+02, 5.92155334e+02, 6.95721209e-01],\n",
       "       [6.01926575e+02, 2.96297577e+02, 6.77372575e-01],\n",
       "       [6.17621460e+02, 4.47154175e+02, 8.15527081e-01],\n",
       "       [6.33207092e+02, 6.13721497e+02, 7.50288665e-01],\n",
       "       [4.49166534e+02, 6.23515259e+02, 3.33123893e-01],\n",
       "       [3.68768433e+02, 6.15664124e+02, 2.96909660e-01],\n",
       "       [0.00000000e+00, 0.00000000e+00, 0.00000000e+00],\n",
       "       [0.00000000e+00, 0.00000000e+00, 0.00000000e+00],\n",
       "       [5.29464417e+02, 6.35266663e+02, 3.00662249e-01],\n",
       "       [0.00000000e+00, 0.00000000e+00, 0.00000000e+00],\n",
       "       [0.00000000e+00, 0.00000000e+00, 0.00000000e+00],\n",
       "       [4.72626495e+02, 1.25848732e+02, 7.93599010e-01],\n",
       "       [5.09897217e+02, 1.25915306e+02, 8.75047982e-01],\n",
       "       [4.47158661e+02, 1.31820786e+02, 9.30340886e-01],\n",
       "       [5.51045410e+02, 1.35715973e+02, 8.41542602e-01],\n",
       "       [0.00000000e+00, 0.00000000e+00, 0.00000000e+00],\n",
       "       [0.00000000e+00, 0.00000000e+00, 0.00000000e+00],\n",
       "       [0.00000000e+00, 0.00000000e+00, 0.00000000e+00],\n",
       "       [0.00000000e+00, 0.00000000e+00, 0.00000000e+00],\n",
       "       [0.00000000e+00, 0.00000000e+00, 0.00000000e+00],\n",
       "       [0.00000000e+00, 0.00000000e+00, 0.00000000e+00]], dtype=float32)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = X.reshape(len(X), 75)\n",
    "dataset[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/200\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 11.8007 - acc: 0.2679\n",
      "Epoch 2/200\n",
      "56/56 [==============================] - 0s 49us/step - loss: 11.7810 - acc: 0.2679\n",
      "Epoch 3/200\n",
      "56/56 [==============================] - 0s 49us/step - loss: 8.9225 - acc: 0.4464\n",
      "Epoch 4/200\n",
      "56/56 [==============================] - 0s 44us/step - loss: 8.3469 - acc: 0.4821\n",
      "Epoch 5/200\n",
      "56/56 [==============================] - 0s 43us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 6/200\n",
      "56/56 [==============================] - 0s 48us/step - loss: 8.3469 - acc: 0.4821\n",
      "Epoch 7/200\n",
      "56/56 [==============================] - 0s 41us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 8/200\n",
      "56/56 [==============================] - 0s 57us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 9/200\n",
      "56/56 [==============================] - 0s 67us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 10/200\n",
      "56/56 [==============================] - 0s 54us/step - loss: 8.3469 - acc: 0.4821\n",
      "Epoch 11/200\n",
      "56/56 [==============================] - 0s 49us/step - loss: 8.9225 - acc: 0.4464\n",
      "Epoch 12/200\n",
      "56/56 [==============================] - 0s 43us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 13/200\n",
      "56/56 [==============================] - 0s 42us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 14/200\n",
      "56/56 [==============================] - 0s 46us/step - loss: 9.2103 - acc: 0.4286\n",
      "Epoch 15/200\n",
      "56/56 [==============================] - 0s 50us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 16/200\n",
      "56/56 [==============================] - 0s 57us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 17/200\n",
      "56/56 [==============================] - 0s 61us/step - loss: 8.9225 - acc: 0.4464\n",
      "Epoch 18/200\n",
      "56/56 [==============================] - 0s 57us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 19/200\n",
      "56/56 [==============================] - 0s 96us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 20/200\n",
      "56/56 [==============================] - 0s 46us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 21/200\n",
      "56/56 [==============================] - 0s 45us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 22/200\n",
      "56/56 [==============================] - 0s 81us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 23/200\n",
      "56/56 [==============================] - 0s 71us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 24/200\n",
      "56/56 [==============================] - 0s 54us/step - loss: 9.2103 - acc: 0.4286\n",
      "Epoch 25/200\n",
      "56/56 [==============================] - 0s 51us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 26/200\n",
      "56/56 [==============================] - 0s 61us/step - loss: 8.9225 - acc: 0.4464\n",
      "Epoch 27/200\n",
      "56/56 [==============================] - 0s 73us/step - loss: 8.8764 - acc: 0.4464\n",
      "Epoch 28/200\n",
      "56/56 [==============================] - 0s 51us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 29/200\n",
      "56/56 [==============================] - 0s 55us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 30/200\n",
      "56/56 [==============================] - 0s 52us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 31/200\n",
      "56/56 [==============================] - 0s 57us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 32/200\n",
      "56/56 [==============================] - 0s 52us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 33/200\n",
      "56/56 [==============================] - 0s 57us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 34/200\n",
      "56/56 [==============================] - 0s 58us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 35/200\n",
      "56/56 [==============================] - 0s 56us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 36/200\n",
      "56/56 [==============================] - 0s 47us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 37/200\n",
      "56/56 [==============================] - 0s 44us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 38/200\n",
      "56/56 [==============================] - 0s 47us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 39/200\n",
      "56/56 [==============================] - 0s 62us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 40/200\n",
      "56/56 [==============================] - 0s 60us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 41/200\n",
      "56/56 [==============================] - 0s 63us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 42/200\n",
      "56/56 [==============================] - 0s 64us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 43/200\n",
      "56/56 [==============================] - 0s 47us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 44/200\n",
      "56/56 [==============================] - 0s 51us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 45/200\n",
      "56/56 [==============================] - 0s 52us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 46/200\n",
      "56/56 [==============================] - 0s 56us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 47/200\n",
      "56/56 [==============================] - 0s 60us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 48/200\n",
      "56/56 [==============================] - 0s 96us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 49/200\n",
      "56/56 [==============================] - 0s 56us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 50/200\n",
      "56/56 [==============================] - 0s 50us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 51/200\n",
      "56/56 [==============================] - 0s 56us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 52/200\n",
      "56/56 [==============================] - 0s 76us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 53/200\n",
      "56/56 [==============================] - 0s 61us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 54/200\n",
      "56/56 [==============================] - 0s 61us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 55/200\n",
      "56/56 [==============================] - 0s 58us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 56/200\n",
      "56/56 [==============================] - 0s 65us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 57/200\n",
      "56/56 [==============================] - 0s 60us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 58/200\n",
      "56/56 [==============================] - 0s 60us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 59/200\n",
      "56/56 [==============================] - 0s 52us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 60/200\n",
      "56/56 [==============================] - 0s 53us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 61/200\n",
      "56/56 [==============================] - 0s 73us/step - loss: 8.9225 - acc: 0.4464\n",
      "Epoch 62/200\n",
      "56/56 [==============================] - 0s 53us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 63/200\n",
      "56/56 [==============================] - 0s 45us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 64/200\n",
      "56/56 [==============================] - 0s 59us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 65/200\n",
      "56/56 [==============================] - 0s 55us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 66/200\n",
      "56/56 [==============================] - 0s 69us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 67/200\n",
      "56/56 [==============================] - 0s 61us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 68/200\n",
      "56/56 [==============================] - 0s 55us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 69/200\n",
      "56/56 [==============================] - 0s 43us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 70/200\n",
      "56/56 [==============================] - 0s 46us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 71/200\n",
      "56/56 [==============================] - 0s 39us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 72/200\n",
      "56/56 [==============================] - 0s 42us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 73/200\n",
      "56/56 [==============================] - ETA: 0s - loss: 8.7038 - acc: 0.460 - 0s 52us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 74/200\n",
      "56/56 [==============================] - 0s 49us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 75/200\n",
      "56/56 [==============================] - 0s 52us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 76/200\n",
      "56/56 [==============================] - 0s 46us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 77/200\n",
      "56/56 [==============================] - 0s 60us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 78/200\n",
      "56/56 [==============================] - 0s 50us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 79/200\n",
      "56/56 [==============================] - 0s 47us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 80/200\n",
      "56/56 [==============================] - 0s 50us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 81/200\n",
      "56/56 [==============================] - 0s 46us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 82/200\n",
      "56/56 [==============================] - 0s 50us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 83/200\n",
      "56/56 [==============================] - 0s 49us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 84/200\n",
      "56/56 [==============================] - 0s 46us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 85/200\n",
      "56/56 [==============================] - 0s 65us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 86/200\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "56/56 [==============================] - 0s 49us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 87/200\n",
      "56/56 [==============================] - 0s 46us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 88/200\n",
      "56/56 [==============================] - 0s 51us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 89/200\n",
      "56/56 [==============================] - 0s 45us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 90/200\n",
      "56/56 [==============================] - 0s 46us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 91/200\n",
      "56/56 [==============================] - 0s 42us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 92/200\n",
      "56/56 [==============================] - 0s 48us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 93/200\n",
      "56/56 [==============================] - 0s 39us/step - loss: 8.3469 - acc: 0.4821\n",
      "Epoch 94/200\n",
      "56/56 [==============================] - 0s 50us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 95/200\n",
      "56/56 [==============================] - 0s 43us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 96/200\n",
      "56/56 [==============================] - 0s 47us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 97/200\n",
      "56/56 [==============================] - 0s 44us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 98/200\n",
      "56/56 [==============================] - 0s 44us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 99/200\n",
      "56/56 [==============================] - 0s 45us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 100/200\n",
      "56/56 [==============================] - 0s 55us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 101/200\n",
      "56/56 [==============================] - 0s 53us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 102/200\n",
      "56/56 [==============================] - 0s 56us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 103/200\n",
      "56/56 [==============================] - 0s 52us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 104/200\n",
      "56/56 [==============================] - 0s 51us/step - loss: 8.9225 - acc: 0.4464\n",
      "Epoch 105/200\n",
      "56/56 [==============================] - 0s 43us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 106/200\n",
      "56/56 [==============================] - 0s 48us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 107/200\n",
      "56/56 [==============================] - 0s 47us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 108/200\n",
      "56/56 [==============================] - 0s 50us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 109/200\n",
      "56/56 [==============================] - 0s 57us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 110/200\n",
      "56/56 [==============================] - 0s 48us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 111/200\n",
      "56/56 [==============================] - 0s 46us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 112/200\n",
      "56/56 [==============================] - 0s 47us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 113/200\n",
      "56/56 [==============================] - 0s 44us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 114/200\n",
      "56/56 [==============================] - 0s 45us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 115/200\n",
      "56/56 [==============================] - 0s 42us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 116/200\n",
      "56/56 [==============================] - 0s 40us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 117/200\n",
      "56/56 [==============================] - 0s 65us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 118/200\n",
      "56/56 [==============================] - 0s 50us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 119/200\n",
      "56/56 [==============================] - 0s 46us/step - loss: 8.9225 - acc: 0.4464\n",
      "Epoch 120/200\n",
      "56/56 [==============================] - 0s 44us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 121/200\n",
      "56/56 [==============================] - 0s 48us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 122/200\n",
      "56/56 [==============================] - 0s 59us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 123/200\n",
      "56/56 [==============================] - 0s 58us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 124/200\n",
      "56/56 [==============================] - 0s 44us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 125/200\n",
      "56/56 [==============================] - 0s 40us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 126/200\n",
      "56/56 [==============================] - 0s 44us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 127/200\n",
      "56/56 [==============================] - 0s 46us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 128/200\n",
      "56/56 [==============================] - 0s 52us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 129/200\n",
      "56/56 [==============================] - 0s 44us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 130/200\n",
      "56/56 [==============================] - 0s 42us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 131/200\n",
      "56/56 [==============================] - 0s 42us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 132/200\n",
      "56/56 [==============================] - 0s 48us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 133/200\n",
      "56/56 [==============================] - 0s 48us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 134/200\n",
      "56/56 [==============================] - 0s 45us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 135/200\n",
      "56/56 [==============================] - 0s 55us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 136/200\n",
      "56/56 [==============================] - 0s 52us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 137/200\n",
      "56/56 [==============================] - 0s 49us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 138/200\n",
      "56/56 [==============================] - 0s 54us/step - loss: 8.3469 - acc: 0.4821\n",
      "Epoch 139/200\n",
      "56/56 [==============================] - 0s 54us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 140/200\n",
      "56/56 [==============================] - 0s 52us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 141/200\n",
      "56/56 [==============================] - 0s 48us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 142/200\n",
      "56/56 [==============================] - 0s 56us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 143/200\n",
      "56/56 [==============================] - 0s 52us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 144/200\n",
      "56/56 [==============================] - 0s 53us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 145/200\n",
      "56/56 [==============================] - 0s 48us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 146/200\n",
      "56/56 [==============================] - 0s 52us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 147/200\n",
      "56/56 [==============================] - 0s 58us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 148/200\n",
      "56/56 [==============================] - 0s 52us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 149/200\n",
      "56/56 [==============================] - 0s 67us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 150/200\n",
      "56/56 [==============================] - 0s 49us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 151/200\n",
      "56/56 [==============================] - 0s 48us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 152/200\n",
      "56/56 [==============================] - 0s 56us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 153/200\n",
      "56/56 [==============================] - 0s 50us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 154/200\n",
      "56/56 [==============================] - 0s 46us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 155/200\n",
      "56/56 [==============================] - 0s 52us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 156/200\n",
      "56/56 [==============================] - 0s 59us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 157/200\n",
      "56/56 [==============================] - 0s 48us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 158/200\n",
      "56/56 [==============================] - 0s 52us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 159/200\n",
      "56/56 [==============================] - 0s 56us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 160/200\n",
      "56/56 [==============================] - 0s 72us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 161/200\n",
      "56/56 [==============================] - 0s 60us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 162/200\n",
      "56/56 [==============================] - 0s 52us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 163/200\n",
      "56/56 [==============================] - 0s 53us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 164/200\n",
      "56/56 [==============================] - 0s 45us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 165/200\n",
      "56/56 [==============================] - 0s 45us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 166/200\n",
      "56/56 [==============================] - 0s 58us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 167/200\n",
      "56/56 [==============================] - 0s 49us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 168/200\n",
      "56/56 [==============================] - 0s 41us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 169/200\n",
      "56/56 [==============================] - 0s 45us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 170/200\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "56/56 [==============================] - 0s 47us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 171/200\n",
      "56/56 [==============================] - 0s 45us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 172/200\n",
      "56/56 [==============================] - 0s 43us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 173/200\n",
      "56/56 [==============================] - 0s 43us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 174/200\n",
      "56/56 [==============================] - 0s 45us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 175/200\n",
      "56/56 [==============================] - 0s 46us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 176/200\n",
      "56/56 [==============================] - 0s 50us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 177/200\n",
      "56/56 [==============================] - 0s 47us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 178/200\n",
      "56/56 [==============================] - 0s 43us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 179/200\n",
      "56/56 [==============================] - 0s 59us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 180/200\n",
      "56/56 [==============================] - 0s 51us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 181/200\n",
      "56/56 [==============================] - 0s 55us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 182/200\n",
      "56/56 [==============================] - 0s 51us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 183/200\n",
      "56/56 [==============================] - 0s 48us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 184/200\n",
      "56/56 [==============================] - 0s 46us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 185/200\n",
      "56/56 [==============================] - 0s 50us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 186/200\n",
      "56/56 [==============================] - 0s 55us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 187/200\n",
      "56/56 [==============================] - 0s 51us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 188/200\n",
      "56/56 [==============================] - 0s 47us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 189/200\n",
      "56/56 [==============================] - 0s 54us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 190/200\n",
      "56/56 [==============================] - 0s 45us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 191/200\n",
      "56/56 [==============================] - 0s 51us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 192/200\n",
      "56/56 [==============================] - 0s 65us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 193/200\n",
      "56/56 [==============================] - 0s 45us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 194/200\n",
      "56/56 [==============================] - 0s 57us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 195/200\n",
      "56/56 [==============================] - 0s 79us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 196/200\n",
      "56/56 [==============================] - 0s 50us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 197/200\n",
      "56/56 [==============================] - 0s 47us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 198/200\n",
      "56/56 [==============================] - 0s 50us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 199/200\n",
      "56/56 [==============================] - 0s 44us/step - loss: 8.6347 - acc: 0.4643\n",
      "Epoch 200/200\n",
      "56/56 [==============================] - 0s 48us/step - loss: 8.6347 - acc: 0.4643\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7fde9e3e9c50>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "opt = SGD(lr=0.005)\n",
    "model = Sequential()\n",
    "model.add(Dense(128, activation='relu', input_shape=(75,)))\n",
    "model.add(Dropout(0.5))\n",
    "model.add(Dense(128, activation='relu'))\n",
    "model.add(Dropout(0.5))\n",
    "model.add(Dense(y.shape[1], activation='softmax'))\n",
    "model.compile(optimizer=opt, #'Adam',\n",
    "              loss='categorical_crossentropy',\n",
    "              metrics=['accuracy'])\n",
    "model.fit(X, y, epochs=200,batch_size=50)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Cleaning up data further\n",
    "\n",
    "Looking at our accuracy, it looks like we need to better prepare and nomalize our data. \n",
    "\n",
    "Or maybe we need to try a different optimizer. Here, I've tried both `SGD` and `Adam`, and saw no improvement."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(56, 25, 3)\n",
      "(56, 25, 2)\n",
      "(56, 50)\n"
     ]
    }
   ],
   "source": [
    "# let's fit x and y to 0 - 1, get rid of confidence and try again\n",
    "X, y = shuffle(dataset, labels)\n",
    "y = to_categorical(y, 3)\n",
    "print(X.shape)\n",
    "X[:,:,0] = X[:,:,0] / 720 # I think the dimensions are 1280 x 720 ?\n",
    "X[:,:,1] = X[:,:,1] / 1280  # let's see?\n",
    "X = X[:,:,1:]\n",
    "print(X.shape)\n",
    "X = X.reshape(56, 50)      # we got rid of confidence percentage\n",
    "print(X.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/200\n",
      "56/56 [==============================] - 0s 3ms/step - loss: 1.1181 - acc: 0.3214\n",
      "Epoch 2/200\n",
      "56/56 [==============================] - 0s 85us/step - loss: 1.1080 - acc: 0.4107\n",
      "Epoch 3/200\n",
      "56/56 [==============================] - 0s 62us/step - loss: 1.0874 - acc: 0.3571\n",
      "Epoch 4/200\n",
      "56/56 [==============================] - 0s 73us/step - loss: 1.0482 - acc: 0.4286\n",
      "Epoch 5/200\n",
      "56/56 [==============================] - 0s 89us/step - loss: 1.0640 - acc: 0.4107\n",
      "Epoch 6/200\n",
      "56/56 [==============================] - 0s 64us/step - loss: 1.0018 - acc: 0.4107\n",
      "Epoch 7/200\n",
      "56/56 [==============================] - 0s 59us/step - loss: 1.0232 - acc: 0.5536\n",
      "Epoch 8/200\n",
      "56/56 [==============================] - 0s 83us/step - loss: 1.0465 - acc: 0.4286\n",
      "Epoch 9/200\n",
      "56/56 [==============================] - 0s 104us/step - loss: 1.0455 - acc: 0.4821\n",
      "Epoch 10/200\n",
      "56/56 [==============================] - 0s 90us/step - loss: 1.0816 - acc: 0.4464\n",
      "Epoch 11/200\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.9902 - acc: 0.5357\n",
      "Epoch 12/200\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.9998 - acc: 0.5179\n",
      "Epoch 13/200\n",
      "56/56 [==============================] - 0s 66us/step - loss: 1.0120 - acc: 0.5536\n",
      "Epoch 14/200\n",
      "56/56 [==============================] - 0s 174us/step - loss: 1.0120 - acc: 0.5536\n",
      "Epoch 15/200\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.9832 - acc: 0.4464\n",
      "Epoch 16/200\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.9401 - acc: 0.5714\n",
      "Epoch 17/200\n",
      "56/56 [==============================] - 0s 79us/step - loss: 0.9692 - acc: 0.5714\n",
      "Epoch 18/200\n",
      "56/56 [==============================] - 0s 95us/step - loss: 0.9571 - acc: 0.4643\n",
      "Epoch 19/200\n",
      "56/56 [==============================] - 0s 82us/step - loss: 0.9630 - acc: 0.5536\n",
      "Epoch 20/200\n",
      "56/56 [==============================] - 0s 64us/step - loss: 0.8602 - acc: 0.6071\n",
      "Epoch 21/200\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.9576 - acc: 0.4821\n",
      "Epoch 22/200\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.9265 - acc: 0.5179\n",
      "Epoch 23/200\n",
      "56/56 [==============================] - 0s 83us/step - loss: 0.8936 - acc: 0.5357\n",
      "Epoch 24/200\n",
      "56/56 [==============================] - 0s 78us/step - loss: 0.9079 - acc: 0.5536\n",
      "Epoch 25/200\n",
      "56/56 [==============================] - 0s 69us/step - loss: 0.8699 - acc: 0.5536\n",
      "Epoch 26/200\n",
      "56/56 [==============================] - 0s 59us/step - loss: 0.9075 - acc: 0.5536\n",
      "Epoch 27/200\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.8978 - acc: 0.5714\n",
      "Epoch 28/200\n",
      "56/56 [==============================] - 0s 82us/step - loss: 0.8842 - acc: 0.6250\n",
      "Epoch 29/200\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.8414 - acc: 0.6429\n",
      "Epoch 30/200\n",
      "56/56 [==============================] - 0s 64us/step - loss: 0.8472 - acc: 0.5714\n",
      "Epoch 31/200\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.8905 - acc: 0.5536\n",
      "Epoch 32/200\n",
      "56/56 [==============================] - 0s 75us/step - loss: 0.8176 - acc: 0.6964\n",
      "Epoch 33/200\n",
      "56/56 [==============================] - 0s 95us/step - loss: 0.7840 - acc: 0.6964\n",
      "Epoch 34/200\n",
      "56/56 [==============================] - 0s 79us/step - loss: 0.7879 - acc: 0.6250\n",
      "Epoch 35/200\n",
      "56/56 [==============================] - 0s 74us/step - loss: 0.8054 - acc: 0.6964\n",
      "Epoch 36/200\n",
      "56/56 [==============================] - 0s 65us/step - loss: 0.8339 - acc: 0.6429\n",
      "Epoch 37/200\n",
      "56/56 [==============================] - 0s 75us/step - loss: 0.8199 - acc: 0.6607\n",
      "Epoch 38/200\n",
      "56/56 [==============================] - 0s 74us/step - loss: 0.8184 - acc: 0.6429\n",
      "Epoch 39/200\n",
      "56/56 [==============================] - 0s 144us/step - loss: 0.7656 - acc: 0.7143\n",
      "Epoch 40/200\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.8403 - acc: 0.6071\n",
      "Epoch 41/200\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.7397 - acc: 0.8036\n",
      "Epoch 42/200\n",
      "56/56 [==============================] - 0s 90us/step - loss: 0.7842 - acc: 0.6607\n",
      "Epoch 43/200\n",
      "56/56 [==============================] - 0s 136us/step - loss: 0.7386 - acc: 0.6786\n",
      "Epoch 44/200\n",
      "56/56 [==============================] - 0s 74us/step - loss: 0.7431 - acc: 0.6786\n",
      "Epoch 45/200\n",
      "56/56 [==============================] - 0s 62us/step - loss: 0.7183 - acc: 0.7679\n",
      "Epoch 46/200\n",
      "56/56 [==============================] - 0s 111us/step - loss: 0.7489 - acc: 0.6786\n",
      "Epoch 47/200\n",
      "56/56 [==============================] - 0s 83us/step - loss: 0.7082 - acc: 0.7500\n",
      "Epoch 48/200\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.7029 - acc: 0.6607\n",
      "Epoch 49/200\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.6830 - acc: 0.7500\n",
      "Epoch 50/200\n",
      "56/56 [==============================] - 0s 69us/step - loss: 0.6353 - acc: 0.7143\n",
      "Epoch 51/200\n",
      "56/56 [==============================] - 0s 67us/step - loss: 0.6603 - acc: 0.7321\n",
      "Epoch 52/200\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.6567 - acc: 0.6964\n",
      "Epoch 53/200\n",
      "56/56 [==============================] - 0s 81us/step - loss: 0.6564 - acc: 0.7143\n",
      "Epoch 54/200\n",
      "56/56 [==============================] - 0s 79us/step - loss: 0.5689 - acc: 0.8036\n",
      "Epoch 55/200\n",
      "56/56 [==============================] - 0s 58us/step - loss: 0.6173 - acc: 0.7143\n",
      "Epoch 56/200\n",
      "56/56 [==============================] - 0s 61us/step - loss: 0.6302 - acc: 0.7321\n",
      "Epoch 57/200\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.6036 - acc: 0.6964\n",
      "Epoch 58/200\n",
      "56/56 [==============================] - 0s 97us/step - loss: 0.5556 - acc: 0.8036\n",
      "Epoch 59/200\n",
      "56/56 [==============================] - 0s 101us/step - loss: 0.6340 - acc: 0.7143\n",
      "Epoch 60/200\n",
      "56/56 [==============================] - 0s 56us/step - loss: 0.6534 - acc: 0.6607\n",
      "Epoch 61/200\n",
      "56/56 [==============================] - 0s 75us/step - loss: 0.6219 - acc: 0.7321\n",
      "Epoch 62/200\n",
      "56/56 [==============================] - 0s 78us/step - loss: 0.5575 - acc: 0.7857\n",
      "Epoch 63/200\n",
      "56/56 [==============================] - 0s 64us/step - loss: 0.5349 - acc: 0.8036\n",
      "Epoch 64/200\n",
      "56/56 [==============================] - 0s 84us/step - loss: 0.5685 - acc: 0.7857\n",
      "Epoch 65/200\n",
      "56/56 [==============================] - 0s 86us/step - loss: 0.5681 - acc: 0.7321\n",
      "Epoch 66/200\n",
      "56/56 [==============================] - 0s 81us/step - loss: 0.5435 - acc: 0.8036\n",
      "Epoch 67/200\n",
      "56/56 [==============================] - 0s 80us/step - loss: 0.4840 - acc: 0.8571\n",
      "Epoch 68/200\n",
      "56/56 [==============================] - 0s 114us/step - loss: 0.5229 - acc: 0.7321\n",
      "Epoch 69/200\n",
      "56/56 [==============================] - 0s 80us/step - loss: 0.5247 - acc: 0.7857\n",
      "Epoch 70/200\n",
      "56/56 [==============================] - 0s 78us/step - loss: 0.5449 - acc: 0.7143\n",
      "Epoch 71/200\n",
      "56/56 [==============================] - 0s 77us/step - loss: 0.5456 - acc: 0.7857\n",
      "Epoch 72/200\n",
      "56/56 [==============================] - 0s 105us/step - loss: 0.4857 - acc: 0.8036\n",
      "Epoch 73/200\n",
      "56/56 [==============================] - 0s 80us/step - loss: 0.5204 - acc: 0.7857\n",
      "Epoch 74/200\n",
      "56/56 [==============================] - 0s 69us/step - loss: 0.5074 - acc: 0.8393\n",
      "Epoch 75/200\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.4894 - acc: 0.8036\n",
      "Epoch 76/200\n",
      "56/56 [==============================] - 0s 69us/step - loss: 0.4819 - acc: 0.7679\n",
      "Epoch 77/200\n",
      "56/56 [==============================] - 0s 84us/step - loss: 0.4925 - acc: 0.7857\n",
      "Epoch 78/200\n",
      "56/56 [==============================] - 0s 79us/step - loss: 0.5051 - acc: 0.7500\n",
      "Epoch 79/200\n",
      "56/56 [==============================] - 0s 74us/step - loss: 0.4937 - acc: 0.7857\n",
      "Epoch 80/200\n",
      "56/56 [==============================] - 0s 65us/step - loss: 0.4562 - acc: 0.8214\n",
      "Epoch 81/200\n",
      "56/56 [==============================] - 0s 89us/step - loss: 0.4478 - acc: 0.7679\n",
      "Epoch 82/200\n",
      "56/56 [==============================] - 0s 61us/step - loss: 0.4057 - acc: 0.8750\n",
      "Epoch 83/200\n",
      "56/56 [==============================] - 0s 61us/step - loss: 0.4634 - acc: 0.7679\n",
      "Epoch 84/200\n",
      "56/56 [==============================] - 0s 67us/step - loss: 0.4733 - acc: 0.7857\n",
      "Epoch 85/200\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.4951 - acc: 0.7321\n",
      "Epoch 86/200\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "56/56 [==============================] - 0s 73us/step - loss: 0.4853 - acc: 0.7500\n",
      "Epoch 87/200\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.4310 - acc: 0.8571\n",
      "Epoch 88/200\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.4120 - acc: 0.8393\n",
      "Epoch 89/200\n",
      "56/56 [==============================] - 0s 83us/step - loss: 0.4127 - acc: 0.8571\n",
      "Epoch 90/200\n",
      "56/56 [==============================] - 0s 81us/step - loss: 0.4381 - acc: 0.8393\n",
      "Epoch 91/200\n",
      "56/56 [==============================] - 0s 81us/step - loss: 0.4103 - acc: 0.8571\n",
      "Epoch 92/200\n",
      "56/56 [==============================] - 0s 74us/step - loss: 0.4100 - acc: 0.8393\n",
      "Epoch 93/200\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.3548 - acc: 0.8750\n",
      "Epoch 94/200\n",
      "56/56 [==============================] - 0s 74us/step - loss: 0.4244 - acc: 0.8750\n",
      "Epoch 95/200\n",
      "56/56 [==============================] - 0s 63us/step - loss: 0.4012 - acc: 0.8393\n",
      "Epoch 96/200\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.3559 - acc: 0.8571\n",
      "Epoch 97/200\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.4632 - acc: 0.8393\n",
      "Epoch 98/200\n",
      "56/56 [==============================] - 0s 81us/step - loss: 0.3885 - acc: 0.8750\n",
      "Epoch 99/200\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.4293 - acc: 0.8214\n",
      "Epoch 100/200\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.3795 - acc: 0.8750\n",
      "Epoch 101/200\n",
      "56/56 [==============================] - 0s 75us/step - loss: 0.4361 - acc: 0.7857\n",
      "Epoch 102/200\n",
      "56/56 [==============================] - 0s 69us/step - loss: 0.3942 - acc: 0.7857\n",
      "Epoch 103/200\n",
      "56/56 [==============================] - 0s 100us/step - loss: 0.3949 - acc: 0.7679\n",
      "Epoch 104/200\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.3640 - acc: 0.8393\n",
      "Epoch 105/200\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.3181 - acc: 0.8571\n",
      "Epoch 106/200\n",
      "56/56 [==============================] - 0s 83us/step - loss: 0.4689 - acc: 0.7321\n",
      "Epoch 107/200\n",
      "56/56 [==============================] - 0s 116us/step - loss: 0.3919 - acc: 0.8929\n",
      "Epoch 108/200\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.4510 - acc: 0.7500\n",
      "Epoch 109/200\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.3673 - acc: 0.8214\n",
      "Epoch 110/200\n",
      "56/56 [==============================] - 0s 87us/step - loss: 0.4017 - acc: 0.8036\n",
      "Epoch 111/200\n",
      "56/56 [==============================] - 0s 64us/step - loss: 0.3938 - acc: 0.8571\n",
      "Epoch 112/200\n",
      "56/56 [==============================] - 0s 59us/step - loss: 0.4166 - acc: 0.8393\n",
      "Epoch 113/200\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.3887 - acc: 0.8214\n",
      "Epoch 114/200\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.3961 - acc: 0.8036\n",
      "Epoch 115/200\n",
      "56/56 [==============================] - 0s 65us/step - loss: 0.3612 - acc: 0.8571\n",
      "Epoch 116/200\n",
      "56/56 [==============================] - 0s 65us/step - loss: 0.4181 - acc: 0.8214\n",
      "Epoch 117/200\n",
      "56/56 [==============================] - 0s 105us/step - loss: 0.4135 - acc: 0.7500\n",
      "Epoch 118/200\n",
      "56/56 [==============================] - 0s 74us/step - loss: 0.3847 - acc: 0.8036\n",
      "Epoch 119/200\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.3388 - acc: 0.8393\n",
      "Epoch 120/200\n",
      "56/56 [==============================] - 0s 75us/step - loss: 0.3682 - acc: 0.8393\n",
      "Epoch 121/200\n",
      "56/56 [==============================] - 0s 92us/step - loss: 0.3899 - acc: 0.8036\n",
      "Epoch 122/200\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.3258 - acc: 0.8750\n",
      "Epoch 123/200\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.3785 - acc: 0.8214\n",
      "Epoch 124/200\n",
      "56/56 [==============================] - 0s 88us/step - loss: 0.3737 - acc: 0.8393\n",
      "Epoch 125/200\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.3596 - acc: 0.8393\n",
      "Epoch 126/200\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.3552 - acc: 0.8393\n",
      "Epoch 127/200\n",
      "56/56 [==============================] - 0s 78us/step - loss: 0.3328 - acc: 0.8750\n",
      "Epoch 128/200\n",
      "56/56 [==============================] - 0s 98us/step - loss: 0.3456 - acc: 0.8929\n",
      "Epoch 129/200\n",
      "56/56 [==============================] - 0s 79us/step - loss: 0.3267 - acc: 0.8571\n",
      "Epoch 130/200\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.3431 - acc: 0.8393\n",
      "Epoch 131/200\n",
      "56/56 [==============================] - 0s 76us/step - loss: 0.3288 - acc: 0.8393\n",
      "Epoch 132/200\n",
      "56/56 [==============================] - 0s 80us/step - loss: 0.3359 - acc: 0.9107\n",
      "Epoch 133/200\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.2904 - acc: 0.8929\n",
      "Epoch 134/200\n",
      "56/56 [==============================] - 0s 84us/step - loss: 0.3375 - acc: 0.8750\n",
      "Epoch 135/200\n",
      "56/56 [==============================] - 0s 67us/step - loss: 0.3342 - acc: 0.8571\n",
      "Epoch 136/200\n",
      "56/56 [==============================] - 0s 65us/step - loss: 0.3219 - acc: 0.8750\n",
      "Epoch 137/200\n",
      "56/56 [==============================] - 0s 74us/step - loss: 0.3117 - acc: 0.8571\n",
      "Epoch 138/200\n",
      "56/56 [==============================] - 0s 81us/step - loss: 0.3151 - acc: 0.8571\n",
      "Epoch 139/200\n",
      "56/56 [==============================] - 0s 63us/step - loss: 0.3133 - acc: 0.8393\n",
      "Epoch 140/200\n",
      "56/56 [==============================] - 0s 76us/step - loss: 0.3086 - acc: 0.8393\n",
      "Epoch 141/200\n",
      "56/56 [==============================] - 0s 79us/step - loss: 0.3999 - acc: 0.7679\n",
      "Epoch 142/200\n",
      "56/56 [==============================] - 0s 69us/step - loss: 0.3015 - acc: 0.8036\n",
      "Epoch 143/200\n",
      "56/56 [==============================] - 0s 63us/step - loss: 0.3159 - acc: 0.8036\n",
      "Epoch 144/200\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.3227 - acc: 0.8393\n",
      "Epoch 145/200\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.2770 - acc: 0.8929\n",
      "Epoch 146/200\n",
      "56/56 [==============================] - 0s 81us/step - loss: 0.3155 - acc: 0.8750\n",
      "Epoch 147/200\n",
      "56/56 [==============================] - ETA: 0s - loss: 0.2043 - acc: 0.920 - 0s 80us/step - loss: 0.3265 - acc: 0.8393\n",
      "Epoch 148/200\n",
      "56/56 [==============================] - 0s 79us/step - loss: 0.3047 - acc: 0.8571\n",
      "Epoch 149/200\n",
      "56/56 [==============================] - 0s 94us/step - loss: 0.3051 - acc: 0.8750\n",
      "Epoch 150/200\n",
      "56/56 [==============================] - 0s 95us/step - loss: 0.3210 - acc: 0.8750\n",
      "Epoch 151/200\n",
      "56/56 [==============================] - 0s 94us/step - loss: 0.3338 - acc: 0.8571\n",
      "Epoch 152/200\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.3108 - acc: 0.8393\n",
      "Epoch 153/200\n",
      "56/56 [==============================] - 0s 67us/step - loss: 0.2752 - acc: 0.8929\n",
      "Epoch 154/200\n",
      "56/56 [==============================] - 0s 93us/step - loss: 0.3082 - acc: 0.8393\n",
      "Epoch 155/200\n",
      "56/56 [==============================] - 0s 67us/step - loss: 0.3121 - acc: 0.8571\n",
      "Epoch 156/200\n",
      "56/56 [==============================] - 0s 60us/step - loss: 0.2853 - acc: 0.8750\n",
      "Epoch 157/200\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.2739 - acc: 0.8750\n",
      "Epoch 158/200\n",
      "56/56 [==============================] - 0s 62us/step - loss: 0.2978 - acc: 0.8571\n",
      "Epoch 159/200\n",
      "56/56 [==============================] - 0s 76us/step - loss: 0.3054 - acc: 0.8393\n",
      "Epoch 160/200\n",
      "56/56 [==============================] - 0s 77us/step - loss: 0.2667 - acc: 0.8571\n",
      "Epoch 161/200\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.2680 - acc: 0.8393\n",
      "Epoch 162/200\n",
      "56/56 [==============================] - 0s 85us/step - loss: 0.2400 - acc: 0.9464\n",
      "Epoch 163/200\n",
      "56/56 [==============================] - 0s 65us/step - loss: 0.2712 - acc: 0.8929\n",
      "Epoch 164/200\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.2761 - acc: 0.8929\n",
      "Epoch 165/200\n",
      "56/56 [==============================] - 0s 75us/step - loss: 0.2661 - acc: 0.8393\n",
      "Epoch 166/200\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.2383 - acc: 0.9107\n",
      "Epoch 167/200\n",
      "56/56 [==============================] - 0s 65us/step - loss: 0.3636 - acc: 0.8036\n",
      "Epoch 168/200\n",
      "56/56 [==============================] - 0s 77us/step - loss: 0.2232 - acc: 0.8929\n",
      "Epoch 169/200\n",
      "56/56 [==============================] - 0s 75us/step - loss: 0.2401 - acc: 0.9464\n",
      "Epoch 170/200\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "56/56 [==============================] - 0s 70us/step - loss: 0.2678 - acc: 0.9107\n",
      "Epoch 171/200\n",
      "56/56 [==============================] - 0s 62us/step - loss: 0.2800 - acc: 0.8929\n",
      "Epoch 172/200\n",
      "56/56 [==============================] - 0s 61us/step - loss: 0.3129 - acc: 0.7857\n",
      "Epoch 173/200\n",
      "56/56 [==============================] - 0s 76us/step - loss: 0.2324 - acc: 0.9107\n",
      "Epoch 174/200\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.2621 - acc: 0.8750\n",
      "Epoch 175/200\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.2920 - acc: 0.8750\n",
      "Epoch 176/200\n",
      "56/56 [==============================] - 0s 64us/step - loss: 0.3364 - acc: 0.8214\n",
      "Epoch 177/200\n",
      "56/56 [==============================] - 0s 84us/step - loss: 0.2525 - acc: 0.9107\n",
      "Epoch 178/200\n",
      "56/56 [==============================] - 0s 60us/step - loss: 0.2810 - acc: 0.8571\n",
      "Epoch 179/200\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.2379 - acc: 0.9107\n",
      "Epoch 180/200\n",
      "56/56 [==============================] - 0s 80us/step - loss: 0.2494 - acc: 0.8929\n",
      "Epoch 181/200\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.2555 - acc: 0.8929\n",
      "Epoch 182/200\n",
      "56/56 [==============================] - 0s 67us/step - loss: 0.2745 - acc: 0.8750\n",
      "Epoch 183/200\n",
      "56/56 [==============================] - 0s 79us/step - loss: 0.2340 - acc: 0.8929\n",
      "Epoch 184/200\n",
      "56/56 [==============================] - 0s 83us/step - loss: 0.2495 - acc: 0.9107\n",
      "Epoch 185/200\n",
      "56/56 [==============================] - 0s 67us/step - loss: 0.2176 - acc: 0.8750\n",
      "Epoch 186/200\n",
      "56/56 [==============================] - 0s 67us/step - loss: 0.2548 - acc: 0.8393\n",
      "Epoch 187/200\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.2709 - acc: 0.8750\n",
      "Epoch 188/200\n",
      "56/56 [==============================] - 0s 94us/step - loss: 0.2596 - acc: 0.8571\n",
      "Epoch 189/200\n",
      "56/56 [==============================] - 0s 65us/step - loss: 0.2723 - acc: 0.8393\n",
      "Epoch 190/200\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.3178 - acc: 0.8750\n",
      "Epoch 191/200\n",
      "56/56 [==============================] - 0s 95us/step - loss: 0.2505 - acc: 0.8929\n",
      "Epoch 192/200\n",
      "56/56 [==============================] - 0s 67us/step - loss: 0.2231 - acc: 0.8750\n",
      "Epoch 193/200\n",
      "56/56 [==============================] - 0s 93us/step - loss: 0.2678 - acc: 0.8929\n",
      "Epoch 194/200\n",
      "56/56 [==============================] - 0s 108us/step - loss: 0.2739 - acc: 0.8393\n",
      "Epoch 195/200\n",
      "56/56 [==============================] - 0s 86us/step - loss: 0.2627 - acc: 0.8750\n",
      "Epoch 196/200\n",
      "56/56 [==============================] - 0s 75us/step - loss: 0.3220 - acc: 0.8393\n",
      "Epoch 197/200\n",
      "56/56 [==============================] - 0s 79us/step - loss: 0.2460 - acc: 0.8929\n",
      "Epoch 198/200\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.2550 - acc: 0.8929\n",
      "Epoch 199/200\n",
      "56/56 [==============================] - 0s 76us/step - loss: 0.2437 - acc: 0.8929\n",
      "Epoch 200/200\n",
      "56/56 [==============================] - 0s 61us/step - loss: 0.2493 - acc: 0.8929\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7fde807e0908>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "opt = SGD(lr=0.01)\n",
    "model = Sequential()\n",
    "model.add(Dense(128, activation='relu', input_shape=(50,)))\n",
    "model.add(Dropout(0.5))\n",
    "model.add(Dense(128, activation='relu'))\n",
    "model.add(Dropout(0.5))\n",
    "model.add(Dense(y.shape[1], activation='softmax'))\n",
    "model.compile(optimizer='Adam',\n",
    "              loss='categorical_crossentropy',\n",
    "              metrics=['accuracy'])\n",
    "model.fit(X, y, epochs=200,batch_size=25)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/500\n",
      "56/56 [==============================] - 0s 4ms/step - loss: 1.1755 - acc: 0.2857\n",
      "Epoch 2/500\n",
      "56/56 [==============================] - 0s 83us/step - loss: 1.1476 - acc: 0.3036\n",
      "Epoch 3/500\n",
      "56/56 [==============================] - 0s 63us/step - loss: 1.1500 - acc: 0.2500\n",
      "Epoch 4/500\n",
      "56/56 [==============================] - 0s 60us/step - loss: 1.1591 - acc: 0.3214\n",
      "Epoch 5/500\n",
      "56/56 [==============================] - 0s 82us/step - loss: 1.1428 - acc: 0.3750\n",
      "Epoch 6/500\n",
      "56/56 [==============================] - 0s 77us/step - loss: 1.1098 - acc: 0.3393\n",
      "Epoch 7/500\n",
      "56/56 [==============================] - 0s 61us/step - loss: 1.1165 - acc: 0.2857\n",
      "Epoch 8/500\n",
      "56/56 [==============================] - 0s 57us/step - loss: 1.1031 - acc: 0.3929\n",
      "Epoch 9/500\n",
      "56/56 [==============================] - 0s 67us/step - loss: 1.0903 - acc: 0.3750\n",
      "Epoch 10/500\n",
      "56/56 [==============================] - 0s 86us/step - loss: 1.0967 - acc: 0.3393\n",
      "Epoch 11/500\n",
      "56/56 [==============================] - 0s 78us/step - loss: 1.0475 - acc: 0.5179\n",
      "Epoch 12/500\n",
      "56/56 [==============================] - 0s 77us/step - loss: 1.0899 - acc: 0.4464\n",
      "Epoch 13/500\n",
      "56/56 [==============================] - 0s 70us/step - loss: 1.1095 - acc: 0.3571\n",
      "Epoch 14/500\n",
      "56/56 [==============================] - 0s 75us/step - loss: 1.1188 - acc: 0.3036\n",
      "Epoch 15/500\n",
      "56/56 [==============================] - 0s 102us/step - loss: 1.0968 - acc: 0.3929\n",
      "Epoch 16/500\n",
      "56/56 [==============================] - 0s 79us/step - loss: 1.0653 - acc: 0.4821\n",
      "Epoch 17/500\n",
      "56/56 [==============================] - 0s 67us/step - loss: 1.1007 - acc: 0.3929\n",
      "Epoch 18/500\n",
      "56/56 [==============================] - 0s 67us/step - loss: 1.0833 - acc: 0.4107\n",
      "Epoch 19/500\n",
      "56/56 [==============================] - 0s 82us/step - loss: 1.0857 - acc: 0.4107\n",
      "Epoch 20/500\n",
      "56/56 [==============================] - 0s 96us/step - loss: 1.0636 - acc: 0.3929\n",
      "Epoch 21/500\n",
      "56/56 [==============================] - 0s 74us/step - loss: 1.0989 - acc: 0.3750\n",
      "Epoch 22/500\n",
      "56/56 [==============================] - 0s 76us/step - loss: 1.0808 - acc: 0.4107\n",
      "Epoch 23/500\n",
      "56/56 [==============================] - 0s 60us/step - loss: 1.0649 - acc: 0.4464\n",
      "Epoch 24/500\n",
      "56/56 [==============================] - 0s 72us/step - loss: 1.0636 - acc: 0.4286\n",
      "Epoch 25/500\n",
      "56/56 [==============================] - 0s 65us/step - loss: 1.0634 - acc: 0.4286\n",
      "Epoch 26/500\n",
      "56/56 [==============================] - 0s 86us/step - loss: 1.0618 - acc: 0.4643\n",
      "Epoch 27/500\n",
      "56/56 [==============================] - 0s 82us/step - loss: 1.0717 - acc: 0.4464\n",
      "Epoch 28/500\n",
      "56/56 [==============================] - 0s 66us/step - loss: 1.0803 - acc: 0.4821\n",
      "Epoch 29/500\n",
      "56/56 [==============================] - 0s 62us/step - loss: 1.0222 - acc: 0.4821\n",
      "Epoch 30/500\n",
      "56/56 [==============================] - 0s 68us/step - loss: 1.0607 - acc: 0.4821\n",
      "Epoch 31/500\n",
      "56/56 [==============================] - 0s 96us/step - loss: 1.0365 - acc: 0.4107\n",
      "Epoch 32/500\n",
      "56/56 [==============================] - 0s 117us/step - loss: 1.0732 - acc: 0.4643\n",
      "Epoch 33/500\n",
      "56/56 [==============================] - 0s 64us/step - loss: 1.0230 - acc: 0.5000\n",
      "Epoch 34/500\n",
      "56/56 [==============================] - 0s 82us/step - loss: 1.0510 - acc: 0.4821\n",
      "Epoch 35/500\n",
      "56/56 [==============================] - 0s 75us/step - loss: 1.0233 - acc: 0.4821\n",
      "Epoch 36/500\n",
      "56/56 [==============================] - 0s 103us/step - loss: 1.0239 - acc: 0.4464\n",
      "Epoch 37/500\n",
      "56/56 [==============================] - 0s 83us/step - loss: 1.0487 - acc: 0.4643\n",
      "Epoch 38/500\n",
      "56/56 [==============================] - 0s 69us/step - loss: 1.0535 - acc: 0.4821\n",
      "Epoch 39/500\n",
      "56/56 [==============================] - 0s 89us/step - loss: 1.0189 - acc: 0.4464\n",
      "Epoch 40/500\n",
      "56/56 [==============================] - 0s 88us/step - loss: 1.0515 - acc: 0.4464\n",
      "Epoch 41/500\n",
      "56/56 [==============================] - 0s 64us/step - loss: 0.9791 - acc: 0.4821\n",
      "Epoch 42/500\n",
      "56/56 [==============================] - 0s 76us/step - loss: 0.9726 - acc: 0.4821\n",
      "Epoch 43/500\n",
      "56/56 [==============================] - 0s 72us/step - loss: 1.0223 - acc: 0.4286\n",
      "Epoch 44/500\n",
      "56/56 [==============================] - 0s 71us/step - loss: 1.0337 - acc: 0.5000\n",
      "Epoch 45/500\n",
      "56/56 [==============================] - 0s 83us/step - loss: 1.0041 - acc: 0.4464\n",
      "Epoch 46/500\n",
      "56/56 [==============================] - 0s 80us/step - loss: 1.0413 - acc: 0.5000\n",
      "Epoch 47/500\n",
      "56/56 [==============================] - 0s 65us/step - loss: 0.9941 - acc: 0.5000\n",
      "Epoch 48/500\n",
      "56/56 [==============================] - 0s 67us/step - loss: 0.9905 - acc: 0.4821\n",
      "Epoch 49/500\n",
      "56/56 [==============================] - 0s 82us/step - loss: 1.0075 - acc: 0.4286\n",
      "Epoch 50/500\n",
      "56/56 [==============================] - 0s 133us/step - loss: 0.9621 - acc: 0.5357\n",
      "Epoch 51/500\n",
      "56/56 [==============================] - 0s 93us/step - loss: 0.9519 - acc: 0.4643\n",
      "Epoch 52/500\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.9440 - acc: 0.5000\n",
      "Epoch 53/500\n",
      "56/56 [==============================] - 0s 170us/step - loss: 1.0151 - acc: 0.4643\n",
      "Epoch 54/500\n",
      "56/56 [==============================] - 0s 96us/step - loss: 0.9175 - acc: 0.5357\n",
      "Epoch 55/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.9544 - acc: 0.5536\n",
      "Epoch 56/500\n",
      "56/56 [==============================] - 0s 61us/step - loss: 0.9433 - acc: 0.5000\n",
      "Epoch 57/500\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.9153 - acc: 0.5536\n",
      "Epoch 58/500\n",
      "56/56 [==============================] - 0s 63us/step - loss: 0.9750 - acc: 0.5000\n",
      "Epoch 59/500\n",
      "56/56 [==============================] - 0s 78us/step - loss: 0.8819 - acc: 0.5893\n",
      "Epoch 60/500\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.9385 - acc: 0.5893\n",
      "Epoch 61/500\n",
      "56/56 [==============================] - 0s 63us/step - loss: 0.8707 - acc: 0.4821\n",
      "Epoch 62/500\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.8819 - acc: 0.5714\n",
      "Epoch 63/500\n",
      "56/56 [==============================] - 0s 118us/step - loss: 0.8938 - acc: 0.5000\n",
      "Epoch 64/500\n",
      "56/56 [==============================] - 0s 80us/step - loss: 0.8841 - acc: 0.5893\n",
      "Epoch 65/500\n",
      "56/56 [==============================] - 0s 59us/step - loss: 0.8665 - acc: 0.5357\n",
      "Epoch 66/500\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.8821 - acc: 0.6250\n",
      "Epoch 67/500\n",
      "56/56 [==============================] - 0s 89us/step - loss: 0.8912 - acc: 0.5714\n",
      "Epoch 68/500\n",
      "56/56 [==============================] - 0s 105us/step - loss: 0.8169 - acc: 0.5536\n",
      "Epoch 69/500\n",
      "56/56 [==============================] - 0s 80us/step - loss: 0.8791 - acc: 0.6250\n",
      "Epoch 70/500\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.7867 - acc: 0.6250\n",
      "Epoch 71/500\n",
      "56/56 [==============================] - 0s 75us/step - loss: 0.8477 - acc: 0.6071\n",
      "Epoch 72/500\n",
      "56/56 [==============================] - 0s 69us/step - loss: 0.7929 - acc: 0.5714\n",
      "Epoch 73/500\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.7660 - acc: 0.6429\n",
      "Epoch 74/500\n",
      "56/56 [==============================] - 0s 95us/step - loss: 0.8239 - acc: 0.6071\n",
      "Epoch 75/500\n",
      "56/56 [==============================] - 0s 86us/step - loss: 0.7885 - acc: 0.5893\n",
      "Epoch 76/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.8372 - acc: 0.5893\n",
      "Epoch 77/500\n",
      "56/56 [==============================] - 0s 77us/step - loss: 0.7358 - acc: 0.6250\n",
      "Epoch 78/500\n",
      "56/56 [==============================] - 0s 78us/step - loss: 0.8114 - acc: 0.5714\n",
      "Epoch 79/500\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.7740 - acc: 0.6071\n",
      "Epoch 80/500\n",
      "56/56 [==============================] - 0s 60us/step - loss: 0.7963 - acc: 0.6607\n",
      "Epoch 81/500\n",
      "56/56 [==============================] - ETA: 0s - loss: 0.8402 - acc: 0.520 - 0s 75us/step - loss: 0.8046 - acc: 0.5357\n",
      "Epoch 82/500\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.7435 - acc: 0.6071\n",
      "Epoch 83/500\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.7187 - acc: 0.6786\n",
      "Epoch 84/500\n",
      "56/56 [==============================] - 0s 79us/step - loss: 0.6943 - acc: 0.6964\n",
      "Epoch 85/500\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.7192 - acc: 0.6964\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 86/500\n",
      "56/56 [==============================] - 0s 65us/step - loss: 0.6595 - acc: 0.6429\n",
      "Epoch 87/500\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.6975 - acc: 0.5893\n",
      "Epoch 88/500\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.6693 - acc: 0.6607\n",
      "Epoch 89/500\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.6520 - acc: 0.7143\n",
      "Epoch 90/500\n",
      "56/56 [==============================] - 0s 69us/step - loss: 0.7347 - acc: 0.6250\n",
      "Epoch 91/500\n",
      "56/56 [==============================] - 0s 60us/step - loss: 0.7615 - acc: 0.6429\n",
      "Epoch 92/500\n",
      "56/56 [==============================] - ETA: 0s - loss: 0.5417 - acc: 0.800 - 0s 82us/step - loss: 0.6381 - acc: 0.7321\n",
      "Epoch 93/500\n",
      "56/56 [==============================] - 0s 78us/step - loss: 0.6086 - acc: 0.7321\n",
      "Epoch 94/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.6893 - acc: 0.6429\n",
      "Epoch 95/500\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.6588 - acc: 0.6607\n",
      "Epoch 96/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.6745 - acc: 0.6250\n",
      "Epoch 97/500\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.6267 - acc: 0.6250\n",
      "Epoch 98/500\n",
      "56/56 [==============================] - 0s 94us/step - loss: 0.5832 - acc: 0.7143\n",
      "Epoch 99/500\n",
      "56/56 [==============================] - 0s 105us/step - loss: 0.6782 - acc: 0.6786\n",
      "Epoch 100/500\n",
      "56/56 [==============================] - 0s 69us/step - loss: 0.6100 - acc: 0.7321\n",
      "Epoch 101/500\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.6015 - acc: 0.6429\n",
      "Epoch 102/500\n",
      "56/56 [==============================] - 0s 80us/step - loss: 0.5603 - acc: 0.7500\n",
      "Epoch 103/500\n",
      "56/56 [==============================] - 0s 107us/step - loss: 0.6363 - acc: 0.6607\n",
      "Epoch 104/500\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.5295 - acc: 0.6964\n",
      "Epoch 105/500\n",
      "56/56 [==============================] - 0s 87us/step - loss: 0.5468 - acc: 0.7143\n",
      "Epoch 106/500\n",
      "56/56 [==============================] - 0s 78us/step - loss: 0.5410 - acc: 0.7321\n",
      "Epoch 107/500\n",
      "56/56 [==============================] - 0s 69us/step - loss: 0.5397 - acc: 0.7679\n",
      "Epoch 108/500\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.5037 - acc: 0.7321\n",
      "Epoch 109/500\n",
      "56/56 [==============================] - 0s 83us/step - loss: 0.6323 - acc: 0.7500\n",
      "Epoch 110/500\n",
      "56/56 [==============================] - 0s 64us/step - loss: 0.6272 - acc: 0.7143\n",
      "Epoch 111/500\n",
      "56/56 [==============================] - 0s 60us/step - loss: 0.5004 - acc: 0.7679\n",
      "Epoch 112/500\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.5375 - acc: 0.7321\n",
      "Epoch 113/500\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.5319 - acc: 0.7143\n",
      "Epoch 114/500\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.6159 - acc: 0.6607\n",
      "Epoch 115/500\n",
      "56/56 [==============================] - 0s 62us/step - loss: 0.5400 - acc: 0.7679\n",
      "Epoch 116/500\n",
      "56/56 [==============================] - 0s 109us/step - loss: 0.5670 - acc: 0.6786\n",
      "Epoch 117/500\n",
      "56/56 [==============================] - 0s 65us/step - loss: 0.5845 - acc: 0.6607\n",
      "Epoch 118/500\n",
      "56/56 [==============================] - 0s 89us/step - loss: 0.5378 - acc: 0.6429\n",
      "Epoch 119/500\n",
      "56/56 [==============================] - 0s 78us/step - loss: 0.5429 - acc: 0.7500\n",
      "Epoch 120/500\n",
      "56/56 [==============================] - 0s 62us/step - loss: 0.4908 - acc: 0.7500\n",
      "Epoch 121/500\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.4948 - acc: 0.7500\n",
      "Epoch 122/500\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.5098 - acc: 0.6607\n",
      "Epoch 123/500\n",
      "56/56 [==============================] - 0s 78us/step - loss: 0.5124 - acc: 0.7321\n",
      "Epoch 124/500\n",
      "56/56 [==============================] - 0s 67us/step - loss: 0.4858 - acc: 0.7143\n",
      "Epoch 125/500\n",
      "56/56 [==============================] - 0s 67us/step - loss: 0.4538 - acc: 0.7679\n",
      "Epoch 126/500\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.5641 - acc: 0.6607\n",
      "Epoch 127/500\n",
      "56/56 [==============================] - 0s 78us/step - loss: 0.5331 - acc: 0.6964\n",
      "Epoch 128/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.4309 - acc: 0.8393\n",
      "Epoch 129/500\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.5923 - acc: 0.6607\n",
      "Epoch 130/500\n",
      "56/56 [==============================] - 0s 87us/step - loss: 0.4722 - acc: 0.7500\n",
      "Epoch 131/500\n",
      "56/56 [==============================] - 0s 60us/step - loss: 0.5756 - acc: 0.6786\n",
      "Epoch 132/500\n",
      "56/56 [==============================] - 0s 81us/step - loss: 0.5129 - acc: 0.7143\n",
      "Epoch 133/500\n",
      "56/56 [==============================] - 0s 69us/step - loss: 0.5125 - acc: 0.6964\n",
      "Epoch 134/500\n",
      "56/56 [==============================] - 0s 82us/step - loss: 0.4874 - acc: 0.6964\n",
      "Epoch 135/500\n",
      "56/56 [==============================] - 0s 61us/step - loss: 0.4757 - acc: 0.7500\n",
      "Epoch 136/500\n",
      "56/56 [==============================] - 0s 60us/step - loss: 0.4563 - acc: 0.7679\n",
      "Epoch 137/500\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.4819 - acc: 0.6607\n",
      "Epoch 138/500\n",
      "56/56 [==============================] - 0s 85us/step - loss: 0.4990 - acc: 0.7500\n",
      "Epoch 139/500\n",
      "56/56 [==============================] - 0s 65us/step - loss: 0.4713 - acc: 0.7143\n",
      "Epoch 140/500\n",
      "56/56 [==============================] - 0s 61us/step - loss: 0.4147 - acc: 0.7500\n",
      "Epoch 141/500\n",
      "56/56 [==============================] - 0s 88us/step - loss: 0.4159 - acc: 0.7857\n",
      "Epoch 142/500\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.4661 - acc: 0.7143\n",
      "Epoch 143/500\n",
      "56/56 [==============================] - 0s 65us/step - loss: 0.4767 - acc: 0.7321\n",
      "Epoch 144/500\n",
      "56/56 [==============================] - 0s 81us/step - loss: 0.4755 - acc: 0.6429\n",
      "Epoch 145/500\n",
      "56/56 [==============================] - 0s 83us/step - loss: 0.5057 - acc: 0.7143\n",
      "Epoch 146/500\n",
      "56/56 [==============================] - 0s 62us/step - loss: 0.4465 - acc: 0.7321\n",
      "Epoch 147/500\n",
      "56/56 [==============================] - 0s 62us/step - loss: 0.4742 - acc: 0.7321\n",
      "Epoch 148/500\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.4228 - acc: 0.8036\n",
      "Epoch 149/500\n",
      "56/56 [==============================] - 0s 76us/step - loss: 0.4376 - acc: 0.7500\n",
      "Epoch 150/500\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.4422 - acc: 0.7679\n",
      "Epoch 151/500\n",
      "56/56 [==============================] - 0s 80us/step - loss: 0.5110 - acc: 0.6786\n",
      "Epoch 152/500\n",
      "56/56 [==============================] - 0s 185us/step - loss: 0.4732 - acc: 0.7321\n",
      "Epoch 153/500\n",
      "56/56 [==============================] - 0s 102us/step - loss: 0.4179 - acc: 0.7321\n",
      "Epoch 154/500\n",
      "56/56 [==============================] - 0s 86us/step - loss: 0.4216 - acc: 0.7679\n",
      "Epoch 155/500\n",
      "56/56 [==============================] - 0s 108us/step - loss: 0.4411 - acc: 0.6607\n",
      "Epoch 156/500\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.4016 - acc: 0.8214\n",
      "Epoch 157/500\n",
      "56/56 [==============================] - 0s 83us/step - loss: 0.4084 - acc: 0.8036\n",
      "Epoch 158/500\n",
      "56/56 [==============================] - 0s 65us/step - loss: 0.4243 - acc: 0.7857\n",
      "Epoch 159/500\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.4396 - acc: 0.7679\n",
      "Epoch 160/500\n",
      "56/56 [==============================] - 0s 82us/step - loss: 0.4175 - acc: 0.7857\n",
      "Epoch 161/500\n",
      "56/56 [==============================] - 0s 81us/step - loss: 0.3928 - acc: 0.7679\n",
      "Epoch 162/500\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.4028 - acc: 0.7679\n",
      "Epoch 163/500\n",
      "56/56 [==============================] - 0s 69us/step - loss: 0.4110 - acc: 0.7679\n",
      "Epoch 164/500\n",
      "56/56 [==============================] - 0s 87us/step - loss: 0.3982 - acc: 0.8393\n",
      "Epoch 165/500\n",
      "56/56 [==============================] - 0s 74us/step - loss: 0.4702 - acc: 0.6607\n",
      "Epoch 166/500\n",
      "56/56 [==============================] - 0s 63us/step - loss: 0.4064 - acc: 0.7857\n",
      "Epoch 167/500\n",
      "56/56 [==============================] - 0s 67us/step - loss: 0.4034 - acc: 0.8214\n",
      "Epoch 168/500\n",
      "56/56 [==============================] - 0s 74us/step - loss: 0.3859 - acc: 0.8036\n",
      "Epoch 169/500\n",
      "56/56 [==============================] - 0s 65us/step - loss: 0.4589 - acc: 0.8036\n",
      "Epoch 170/500\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "56/56 [==============================] - 0s 71us/step - loss: 0.4366 - acc: 0.7321\n",
      "Epoch 171/500\n",
      "56/56 [==============================] - 0s 75us/step - loss: 0.3971 - acc: 0.7857\n",
      "Epoch 172/500\n",
      "56/56 [==============================] - 0s 75us/step - loss: 0.3896 - acc: 0.8214\n",
      "Epoch 173/500\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.4260 - acc: 0.6607\n",
      "Epoch 174/500\n",
      "56/56 [==============================] - 0s 63us/step - loss: 0.3747 - acc: 0.7857\n",
      "Epoch 175/500\n",
      "56/56 [==============================] - 0s 80us/step - loss: 0.4116 - acc: 0.7679\n",
      "Epoch 176/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.3785 - acc: 0.8214\n",
      "Epoch 177/500\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.3909 - acc: 0.8214\n",
      "Epoch 178/500\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.4703 - acc: 0.7500\n",
      "Epoch 179/500\n",
      "56/56 [==============================] - 0s 78us/step - loss: 0.4011 - acc: 0.8393\n",
      "Epoch 180/500\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.3990 - acc: 0.7679\n",
      "Epoch 181/500\n",
      "56/56 [==============================] - 0s 64us/step - loss: 0.3578 - acc: 0.8214\n",
      "Epoch 182/500\n",
      "56/56 [==============================] - 0s 61us/step - loss: 0.3732 - acc: 0.8750\n",
      "Epoch 183/500\n",
      "56/56 [==============================] - 0s 74us/step - loss: 0.4119 - acc: 0.8036\n",
      "Epoch 184/500\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.3946 - acc: 0.7500\n",
      "Epoch 185/500\n",
      "56/56 [==============================] - 0s 64us/step - loss: 0.3760 - acc: 0.8036\n",
      "Epoch 186/500\n",
      "56/56 [==============================] - 0s 65us/step - loss: 0.4313 - acc: 0.7321\n",
      "Epoch 187/500\n",
      "56/56 [==============================] - 0s 78us/step - loss: 0.4135 - acc: 0.7857\n",
      "Epoch 188/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.4383 - acc: 0.7857\n",
      "Epoch 189/500\n",
      "56/56 [==============================] - 0s 79us/step - loss: 0.3905 - acc: 0.8571\n",
      "Epoch 190/500\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.3636 - acc: 0.8750\n",
      "Epoch 191/500\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.3840 - acc: 0.7857\n",
      "Epoch 192/500\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.4142 - acc: 0.7857\n",
      "Epoch 193/500\n",
      "56/56 [==============================] - 0s 67us/step - loss: 0.3668 - acc: 0.7679\n",
      "Epoch 194/500\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.4258 - acc: 0.6964\n",
      "Epoch 195/500\n",
      "56/56 [==============================] - 0s 60us/step - loss: 0.3944 - acc: 0.7679\n",
      "Epoch 196/500\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.4621 - acc: 0.7500\n",
      "Epoch 197/500\n",
      "56/56 [==============================] - 0s 128us/step - loss: 0.3835 - acc: 0.7857\n",
      "Epoch 198/500\n",
      "56/56 [==============================] - 0s 65us/step - loss: 0.4323 - acc: 0.8214\n",
      "Epoch 199/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.3913 - acc: 0.7857\n",
      "Epoch 200/500\n",
      "56/56 [==============================] - 0s 64us/step - loss: 0.3751 - acc: 0.8750\n",
      "Epoch 201/500\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.3888 - acc: 0.7857\n",
      "Epoch 202/500\n",
      "56/56 [==============================] - 0s 77us/step - loss: 0.4191 - acc: 0.7500\n",
      "Epoch 203/500\n",
      "56/56 [==============================] - 0s 78us/step - loss: 0.3550 - acc: 0.8571\n",
      "Epoch 204/500\n",
      "56/56 [==============================] - 0s 80us/step - loss: 0.4581 - acc: 0.8036\n",
      "Epoch 205/500\n",
      "56/56 [==============================] - 0s 98us/step - loss: 0.3820 - acc: 0.8036\n",
      "Epoch 206/500\n",
      "56/56 [==============================] - 0s 62us/step - loss: 0.4941 - acc: 0.7321\n",
      "Epoch 207/500\n",
      "56/56 [==============================] - ETA: 0s - loss: 0.5370 - acc: 0.720 - 0s 78us/step - loss: 0.4433 - acc: 0.7500\n",
      "Epoch 208/500\n",
      "56/56 [==============================] - 0s 65us/step - loss: 0.4087 - acc: 0.7679\n",
      "Epoch 209/500\n",
      "56/56 [==============================] - 0s 87us/step - loss: 0.3692 - acc: 0.7500\n",
      "Epoch 210/500\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.3868 - acc: 0.7857\n",
      "Epoch 211/500\n",
      "56/56 [==============================] - 0s 88us/step - loss: 0.4678 - acc: 0.7143\n",
      "Epoch 212/500\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.4025 - acc: 0.8571\n",
      "Epoch 213/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.3768 - acc: 0.7679\n",
      "Epoch 214/500\n",
      "56/56 [==============================] - 0s 88us/step - loss: 0.3466 - acc: 0.8750\n",
      "Epoch 215/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.4051 - acc: 0.7857\n",
      "Epoch 216/500\n",
      "56/56 [==============================] - 0s 67us/step - loss: 0.3481 - acc: 0.8393\n",
      "Epoch 217/500\n",
      "56/56 [==============================] - 0s 77us/step - loss: 0.4529 - acc: 0.7143\n",
      "Epoch 218/500\n",
      "56/56 [==============================] - 0s 76us/step - loss: 0.3589 - acc: 0.7857\n",
      "Epoch 219/500\n",
      "56/56 [==============================] - 0s 65us/step - loss: 0.3879 - acc: 0.7679\n",
      "Epoch 220/500\n",
      "56/56 [==============================] - 0s 65us/step - loss: 0.3362 - acc: 0.8214\n",
      "Epoch 221/500\n",
      "56/56 [==============================] - 0s 69us/step - loss: 0.3615 - acc: 0.8036\n",
      "Epoch 222/500\n",
      "56/56 [==============================] - 0s 75us/step - loss: 0.3573 - acc: 0.8393\n",
      "Epoch 223/500\n",
      "56/56 [==============================] - 0s 69us/step - loss: 0.3705 - acc: 0.7500\n",
      "Epoch 224/500\n",
      "56/56 [==============================] - 0s 75us/step - loss: 0.3850 - acc: 0.7679\n",
      "Epoch 225/500\n",
      "56/56 [==============================] - 0s 90us/step - loss: 0.3312 - acc: 0.8571\n",
      "Epoch 226/500\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.4123 - acc: 0.7321\n",
      "Epoch 227/500\n",
      "56/56 [==============================] - 0s 74us/step - loss: 0.3750 - acc: 0.7857\n",
      "Epoch 228/500\n",
      "56/56 [==============================] - 0s 74us/step - loss: 0.3573 - acc: 0.7857\n",
      "Epoch 229/500\n",
      "56/56 [==============================] - 0s 78us/step - loss: 0.3605 - acc: 0.8214\n",
      "Epoch 230/500\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.3457 - acc: 0.8036\n",
      "Epoch 231/500\n",
      "56/56 [==============================] - 0s 65us/step - loss: 0.3445 - acc: 0.7679\n",
      "Epoch 232/500\n",
      "56/56 [==============================] - 0s 60us/step - loss: 0.3538 - acc: 0.7857\n",
      "Epoch 233/500\n",
      "56/56 [==============================] - 0s 83us/step - loss: 0.3624 - acc: 0.7679\n",
      "Epoch 234/500\n",
      "56/56 [==============================] - 0s 60us/step - loss: 0.4148 - acc: 0.6964\n",
      "Epoch 235/500\n",
      "56/56 [==============================] - 0s 57us/step - loss: 0.3780 - acc: 0.7321\n",
      "Epoch 236/500\n",
      "56/56 [==============================] - 0s 76us/step - loss: 0.3528 - acc: 0.7857\n",
      "Epoch 237/500\n",
      "56/56 [==============================] - 0s 93us/step - loss: 0.3867 - acc: 0.8214\n",
      "Epoch 238/500\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.3174 - acc: 0.8393\n",
      "Epoch 239/500\n",
      "56/56 [==============================] - 0s 112us/step - loss: 0.4054 - acc: 0.7500\n",
      "Epoch 240/500\n",
      "56/56 [==============================] - 0s 84us/step - loss: 0.4308 - acc: 0.7321\n",
      "Epoch 241/500\n",
      "56/56 [==============================] - 0s 63us/step - loss: 0.3166 - acc: 0.8571\n",
      "Epoch 242/500\n",
      "56/56 [==============================] - 0s 77us/step - loss: 0.3818 - acc: 0.8036\n",
      "Epoch 243/500\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.3704 - acc: 0.7857\n",
      "Epoch 244/500\n",
      "56/56 [==============================] - 0s 84us/step - loss: 0.3400 - acc: 0.8036\n",
      "Epoch 245/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.4424 - acc: 0.7857\n",
      "Epoch 246/500\n",
      "56/56 [==============================] - 0s 92us/step - loss: 0.3289 - acc: 0.7857\n",
      "Epoch 247/500\n",
      "56/56 [==============================] - 0s 64us/step - loss: 0.3683 - acc: 0.7679\n",
      "Epoch 248/500\n",
      "56/56 [==============================] - 0s 75us/step - loss: 0.3551 - acc: 0.7500\n",
      "Epoch 249/500\n",
      "56/56 [==============================] - 0s 85us/step - loss: 0.3318 - acc: 0.8929\n",
      "Epoch 250/500\n",
      "56/56 [==============================] - 0s 63us/step - loss: 0.3319 - acc: 0.8036\n",
      "Epoch 251/500\n",
      "56/56 [==============================] - 0s 76us/step - loss: 0.3576 - acc: 0.8036\n",
      "Epoch 252/500\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.3520 - acc: 0.8214\n",
      "Epoch 253/500\n",
      "56/56 [==============================] - 0s 87us/step - loss: 0.3614 - acc: 0.7500\n",
      "Epoch 254/500\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "56/56 [==============================] - 0s 77us/step - loss: 0.3264 - acc: 0.8036\n",
      "Epoch 255/500\n",
      "56/56 [==============================] - 0s 69us/step - loss: 0.2954 - acc: 0.8571\n",
      "Epoch 256/500\n",
      "56/56 [==============================] - 0s 112us/step - loss: 0.3095 - acc: 0.8214\n",
      "Epoch 257/500\n",
      "56/56 [==============================] - 0s 79us/step - loss: 0.3491 - acc: 0.7500\n",
      "Epoch 258/500\n",
      "56/56 [==============================] - 0s 65us/step - loss: 0.3050 - acc: 0.8036\n",
      "Epoch 259/500\n",
      "56/56 [==============================] - 0s 86us/step - loss: 0.3773 - acc: 0.7679\n",
      "Epoch 260/500\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.3345 - acc: 0.7857\n",
      "Epoch 261/500\n",
      "56/56 [==============================] - 0s 62us/step - loss: 0.3123 - acc: 0.8036\n",
      "Epoch 262/500\n",
      "56/56 [==============================] - 0s 63us/step - loss: 0.3070 - acc: 0.8036\n",
      "Epoch 263/500\n",
      "56/56 [==============================] - 0s 77us/step - loss: 0.3079 - acc: 0.8393\n",
      "Epoch 264/500\n",
      "56/56 [==============================] - 0s 60us/step - loss: 0.3604 - acc: 0.8393\n",
      "Epoch 265/500\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.3599 - acc: 0.8393\n",
      "Epoch 266/500\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.3568 - acc: 0.7857\n",
      "Epoch 267/500\n",
      "56/56 [==============================] - 0s 83us/step - loss: 0.3539 - acc: 0.7321\n",
      "Epoch 268/500\n",
      "56/56 [==============================] - 0s 58us/step - loss: 0.3065 - acc: 0.8036\n",
      "Epoch 269/500\n",
      "56/56 [==============================] - 0s 69us/step - loss: 0.3454 - acc: 0.7679\n",
      "Epoch 270/500\n",
      "56/56 [==============================] - 0s 83us/step - loss: 0.3172 - acc: 0.8214\n",
      "Epoch 271/500\n",
      "56/56 [==============================] - 0s 75us/step - loss: 0.3564 - acc: 0.7857\n",
      "Epoch 272/500\n",
      "56/56 [==============================] - 0s 69us/step - loss: 0.3110 - acc: 0.8750\n",
      "Epoch 273/500\n",
      "56/56 [==============================] - 0s 69us/step - loss: 0.3206 - acc: 0.7857\n",
      "Epoch 274/500\n",
      "56/56 [==============================] - 0s 80us/step - loss: 0.3492 - acc: 0.8393\n",
      "Epoch 275/500\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.2998 - acc: 0.8750\n",
      "Epoch 276/500\n",
      "56/56 [==============================] - 0s 69us/step - loss: 0.3074 - acc: 0.8036\n",
      "Epoch 277/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.3839 - acc: 0.8036\n",
      "Epoch 278/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.3918 - acc: 0.7679\n",
      "Epoch 279/500\n",
      "56/56 [==============================] - 0s 63us/step - loss: 0.3581 - acc: 0.8393\n",
      "Epoch 280/500\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.3640 - acc: 0.8571\n",
      "Epoch 281/500\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.3120 - acc: 0.8036\n",
      "Epoch 282/500\n",
      "56/56 [==============================] - 0s 65us/step - loss: 0.2886 - acc: 0.8571\n",
      "Epoch 283/500\n",
      "56/56 [==============================] - 0s 64us/step - loss: 0.3390 - acc: 0.8214\n",
      "Epoch 284/500\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.2645 - acc: 0.8571\n",
      "Epoch 285/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.3269 - acc: 0.7679\n",
      "Epoch 286/500\n",
      "56/56 [==============================] - 0s 69us/step - loss: 0.3346 - acc: 0.8036\n",
      "Epoch 287/500\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.3597 - acc: 0.8214\n",
      "Epoch 288/500\n",
      "56/56 [==============================] - 0s 63us/step - loss: 0.2999 - acc: 0.8214\n",
      "Epoch 289/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.3182 - acc: 0.8214\n",
      "Epoch 290/500\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.2691 - acc: 0.8750\n",
      "Epoch 291/500\n",
      "56/56 [==============================] - 0s 60us/step - loss: 0.3020 - acc: 0.8036\n",
      "Epoch 292/500\n",
      "56/56 [==============================] - 0s 77us/step - loss: 0.2965 - acc: 0.8571\n",
      "Epoch 293/500\n",
      "56/56 [==============================] - 0s 102us/step - loss: 0.3220 - acc: 0.8036\n",
      "Epoch 294/500\n",
      "56/56 [==============================] - 0s 63us/step - loss: 0.2877 - acc: 0.8036\n",
      "Epoch 295/500\n",
      "56/56 [==============================] - 0s 63us/step - loss: 0.3140 - acc: 0.8750\n",
      "Epoch 296/500\n",
      "56/56 [==============================] - 0s 63us/step - loss: 0.3046 - acc: 0.8393\n",
      "Epoch 297/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.2353 - acc: 0.8571\n",
      "Epoch 298/500\n",
      "56/56 [==============================] - 0s 69us/step - loss: 0.3488 - acc: 0.7857\n",
      "Epoch 299/500\n",
      "56/56 [==============================] - 0s 75us/step - loss: 0.3311 - acc: 0.8036\n",
      "Epoch 300/500\n",
      "56/56 [==============================] - 0s 85us/step - loss: 0.2961 - acc: 0.8036\n",
      "Epoch 301/500\n",
      "56/56 [==============================] - 0s 67us/step - loss: 0.2847 - acc: 0.8571\n",
      "Epoch 302/500\n",
      "56/56 [==============================] - 0s 94us/step - loss: 0.3265 - acc: 0.8036\n",
      "Epoch 303/500\n",
      "56/56 [==============================] - 0s 83us/step - loss: 0.2860 - acc: 0.8571\n",
      "Epoch 304/500\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.2930 - acc: 0.8750\n",
      "Epoch 305/500\n",
      "56/56 [==============================] - 0s 84us/step - loss: 0.3169 - acc: 0.7857\n",
      "Epoch 306/500\n",
      "56/56 [==============================] - 0s 63us/step - loss: 0.3069 - acc: 0.8214\n",
      "Epoch 307/500\n",
      "56/56 [==============================] - 0s 78us/step - loss: 0.2814 - acc: 0.8571\n",
      "Epoch 308/500\n",
      "56/56 [==============================] - 0s 74us/step - loss: 0.3451 - acc: 0.7857\n",
      "Epoch 309/500\n",
      "56/56 [==============================] - 0s 62us/step - loss: 0.2717 - acc: 0.8571\n",
      "Epoch 310/500\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.2609 - acc: 0.8214\n",
      "Epoch 311/500\n",
      "56/56 [==============================] - 0s 83us/step - loss: 0.3469 - acc: 0.8214\n",
      "Epoch 312/500\n",
      "56/56 [==============================] - 0s 74us/step - loss: 0.2853 - acc: 0.8036\n",
      "Epoch 313/500\n",
      "56/56 [==============================] - 0s 74us/step - loss: 0.3254 - acc: 0.8214\n",
      "Epoch 314/500\n",
      "56/56 [==============================] - 0s 83us/step - loss: 0.3169 - acc: 0.8571\n",
      "Epoch 315/500\n",
      "56/56 [==============================] - 0s 59us/step - loss: 0.3455 - acc: 0.7857\n",
      "Epoch 316/500\n",
      "56/56 [==============================] - 0s 67us/step - loss: 0.2923 - acc: 0.8214\n",
      "Epoch 317/500\n",
      "56/56 [==============================] - 0s 63us/step - loss: 0.2799 - acc: 0.8393\n",
      "Epoch 318/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.2910 - acc: 0.8571\n",
      "Epoch 319/500\n",
      "56/56 [==============================] - 0s 61us/step - loss: 0.3132 - acc: 0.8929\n",
      "Epoch 320/500\n",
      "56/56 [==============================] - 0s 60us/step - loss: 0.3381 - acc: 0.7857\n",
      "Epoch 321/500\n",
      "56/56 [==============================] - 0s 61us/step - loss: 0.2963 - acc: 0.8393\n",
      "Epoch 322/500\n",
      "56/56 [==============================] - 0s 80us/step - loss: 0.2639 - acc: 0.8571\n",
      "Epoch 323/500\n",
      "56/56 [==============================] - 0s 87us/step - loss: 0.2803 - acc: 0.8571\n",
      "Epoch 324/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.3418 - acc: 0.7857\n",
      "Epoch 325/500\n",
      "56/56 [==============================] - 0s 86us/step - loss: 0.3006 - acc: 0.8571\n",
      "Epoch 326/500\n",
      "56/56 [==============================] - 0s 60us/step - loss: 0.2915 - acc: 0.8571\n",
      "Epoch 327/500\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.3030 - acc: 0.8393\n",
      "Epoch 328/500\n",
      "56/56 [==============================] - 0s 77us/step - loss: 0.4071 - acc: 0.7679\n",
      "Epoch 329/500\n",
      "56/56 [==============================] - 0s 79us/step - loss: 0.3232 - acc: 0.8214\n",
      "Epoch 330/500\n",
      "56/56 [==============================] - 0s 65us/step - loss: 0.3049 - acc: 0.8750\n",
      "Epoch 331/500\n",
      "56/56 [==============================] - 0s 88us/step - loss: 0.2912 - acc: 0.8393\n",
      "Epoch 332/500\n",
      "56/56 [==============================] - 0s 93us/step - loss: 0.2399 - acc: 0.9286\n",
      "Epoch 333/500\n",
      "56/56 [==============================] - 0s 60us/step - loss: 0.2587 - acc: 0.9107\n",
      "Epoch 334/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.3410 - acc: 0.7857\n",
      "Epoch 335/500\n",
      "56/56 [==============================] - 0s 77us/step - loss: 0.2981 - acc: 0.8214\n",
      "Epoch 336/500\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.3115 - acc: 0.8571\n",
      "Epoch 337/500\n",
      "56/56 [==============================] - 0s 87us/step - loss: 0.2798 - acc: 0.8214\n",
      "Epoch 338/500\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "56/56 [==============================] - 0s 73us/step - loss: 0.2902 - acc: 0.8393\n",
      "Epoch 339/500\n",
      "56/56 [==============================] - 0s 98us/step - loss: 0.3000 - acc: 0.8393\n",
      "Epoch 340/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.3063 - acc: 0.8571\n",
      "Epoch 341/500\n",
      "56/56 [==============================] - 0s 76us/step - loss: 0.2626 - acc: 0.8750\n",
      "Epoch 342/500\n",
      "56/56 [==============================] - 0s 82us/step - loss: 0.2693 - acc: 0.8214\n",
      "Epoch 343/500\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.2581 - acc: 0.8571\n",
      "Epoch 344/500\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.2955 - acc: 0.8393\n",
      "Epoch 345/500\n",
      "56/56 [==============================] - 0s 82us/step - loss: 0.2246 - acc: 0.9107\n",
      "Epoch 346/500\n",
      "56/56 [==============================] - 0s 64us/step - loss: 0.2856 - acc: 0.8393\n",
      "Epoch 347/500\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.2447 - acc: 0.8929\n",
      "Epoch 348/500\n",
      "56/56 [==============================] - 0s 60us/step - loss: 0.2551 - acc: 0.8571\n",
      "Epoch 349/500\n",
      "56/56 [==============================] - 0s 76us/step - loss: 0.2870 - acc: 0.8393\n",
      "Epoch 350/500\n",
      "56/56 [==============================] - 0s 64us/step - loss: 0.3248 - acc: 0.8214\n",
      "Epoch 351/500\n",
      "56/56 [==============================] - 0s 63us/step - loss: 0.2676 - acc: 0.8393\n",
      "Epoch 352/500\n",
      "56/56 [==============================] - 0s 63us/step - loss: 0.2407 - acc: 0.9107\n",
      "Epoch 353/500\n",
      "56/56 [==============================] - 0s 85us/step - loss: 0.2836 - acc: 0.8750\n",
      "Epoch 354/500\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.2807 - acc: 0.9107\n",
      "Epoch 355/500\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.2762 - acc: 0.9107\n",
      "Epoch 356/500\n",
      "56/56 [==============================] - 0s 102us/step - loss: 0.3258 - acc: 0.7857\n",
      "Epoch 357/500\n",
      "56/56 [==============================] - 0s 69us/step - loss: 0.3084 - acc: 0.8571\n",
      "Epoch 358/500\n",
      "56/56 [==============================] - 0s 127us/step - loss: 0.2742 - acc: 0.8571\n",
      "Epoch 359/500\n",
      "56/56 [==============================] - 0s 90us/step - loss: 0.2687 - acc: 0.8571\n",
      "Epoch 360/500\n",
      "56/56 [==============================] - 0s 82us/step - loss: 0.2545 - acc: 0.8750\n",
      "Epoch 361/500\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.2891 - acc: 0.8750\n",
      "Epoch 362/500\n",
      "56/56 [==============================] - 0s 79us/step - loss: 0.2638 - acc: 0.8571\n",
      "Epoch 363/500\n",
      "56/56 [==============================] - 0s 108us/step - loss: 0.2981 - acc: 0.8393\n",
      "Epoch 364/500\n",
      "56/56 [==============================] - 0s 76us/step - loss: 0.2773 - acc: 0.8571\n",
      "Epoch 365/500\n",
      "56/56 [==============================] - 0s 112us/step - loss: 0.2410 - acc: 0.9286\n",
      "Epoch 366/500\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.2555 - acc: 0.8393\n",
      "Epoch 367/500\n",
      "56/56 [==============================] - 0s 85us/step - loss: 0.2420 - acc: 0.8929\n",
      "Epoch 368/500\n",
      "56/56 [==============================] - 0s 102us/step - loss: 0.2732 - acc: 0.8929\n",
      "Epoch 369/500\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.2585 - acc: 0.8750\n",
      "Epoch 370/500\n",
      "56/56 [==============================] - 0s 78us/step - loss: 0.2643 - acc: 0.8214\n",
      "Epoch 371/500\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.2331 - acc: 0.9107\n",
      "Epoch 372/500\n",
      "56/56 [==============================] - 0s 82us/step - loss: 0.2225 - acc: 0.8750\n",
      "Epoch 373/500\n",
      "56/56 [==============================] - 0s 75us/step - loss: 0.2331 - acc: 0.8750\n",
      "Epoch 374/500\n",
      "56/56 [==============================] - 0s 82us/step - loss: 0.2059 - acc: 0.9286\n",
      "Epoch 375/500\n",
      "56/56 [==============================] - 0s 96us/step - loss: 0.2460 - acc: 0.8393\n",
      "Epoch 376/500\n",
      "56/56 [==============================] - 0s 87us/step - loss: 0.2238 - acc: 0.9107\n",
      "Epoch 377/500\n",
      "56/56 [==============================] - 0s 116us/step - loss: 0.2136 - acc: 0.8929\n",
      "Epoch 378/500\n",
      "56/56 [==============================] - 0s 78us/step - loss: 0.2201 - acc: 0.9107\n",
      "Epoch 379/500\n",
      "56/56 [==============================] - 0s 80us/step - loss: 0.2931 - acc: 0.8929\n",
      "Epoch 380/500\n",
      "56/56 [==============================] - 0s 76us/step - loss: 0.3261 - acc: 0.8393\n",
      "Epoch 381/500\n",
      "56/56 [==============================] - 0s 87us/step - loss: 0.2851 - acc: 0.8750\n",
      "Epoch 382/500\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.2652 - acc: 0.8571\n",
      "Epoch 383/500\n",
      "56/56 [==============================] - 0s 79us/step - loss: 0.2867 - acc: 0.8214\n",
      "Epoch 384/500\n",
      "56/56 [==============================] - 0s 91us/step - loss: 0.2601 - acc: 0.8929\n",
      "Epoch 385/500\n",
      "56/56 [==============================] - 0s 87us/step - loss: 0.2789 - acc: 0.8571\n",
      "Epoch 386/500\n",
      "56/56 [==============================] - 0s 89us/step - loss: 0.3367 - acc: 0.8571\n",
      "Epoch 387/500\n",
      "56/56 [==============================] - 0s 81us/step - loss: 0.2679 - acc: 0.8929\n",
      "Epoch 388/500\n",
      "56/56 [==============================] - 0s 62us/step - loss: 0.2137 - acc: 0.8929\n",
      "Epoch 389/500\n",
      "56/56 [==============================] - 0s 75us/step - loss: 0.2869 - acc: 0.8214\n",
      "Epoch 390/500\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.2472 - acc: 0.8571\n",
      "Epoch 391/500\n",
      "56/56 [==============================] - 0s 77us/step - loss: 0.2111 - acc: 0.9107\n",
      "Epoch 392/500\n",
      "56/56 [==============================] - 0s 62us/step - loss: 0.2059 - acc: 0.9286\n",
      "Epoch 393/500\n",
      "56/56 [==============================] - 0s 64us/step - loss: 0.2934 - acc: 0.8393\n",
      "Epoch 394/500\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.2349 - acc: 0.8929\n",
      "Epoch 395/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.2402 - acc: 0.8571\n",
      "Epoch 396/500\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.2306 - acc: 0.8750\n",
      "Epoch 397/500\n",
      "56/56 [==============================] - 0s 69us/step - loss: 0.2039 - acc: 0.9464\n",
      "Epoch 398/500\n",
      "56/56 [==============================] - 0s 79us/step - loss: 0.1978 - acc: 0.9286\n",
      "Epoch 399/500\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.1587 - acc: 0.9464\n",
      "Epoch 400/500\n",
      "56/56 [==============================] - 0s 101us/step - loss: 0.2660 - acc: 0.8571\n",
      "Epoch 401/500\n",
      "56/56 [==============================] - 0s 125us/step - loss: 0.2881 - acc: 0.8214\n",
      "Epoch 402/500\n",
      "56/56 [==============================] - 0s 118us/step - loss: 0.2824 - acc: 0.8214\n",
      "Epoch 403/500\n",
      "56/56 [==============================] - 0s 83us/step - loss: 0.2414 - acc: 0.8929\n",
      "Epoch 404/500\n",
      "56/56 [==============================] - 0s 89us/step - loss: 0.2304 - acc: 0.8393\n",
      "Epoch 405/500\n",
      "56/56 [==============================] - 0s 93us/step - loss: 0.1763 - acc: 0.9821\n",
      "Epoch 406/500\n",
      "56/56 [==============================] - 0s 82us/step - loss: 0.2372 - acc: 0.8393\n",
      "Epoch 407/500\n",
      "56/56 [==============================] - 0s 79us/step - loss: 0.2548 - acc: 0.8214\n",
      "Epoch 408/500\n",
      "56/56 [==============================] - 0s 78us/step - loss: 0.2523 - acc: 0.8929\n",
      "Epoch 409/500\n",
      "56/56 [==============================] - 0s 75us/step - loss: 0.2572 - acc: 0.8214\n",
      "Epoch 410/500\n",
      "56/56 [==============================] - 0s 105us/step - loss: 0.2446 - acc: 0.8929\n",
      "Epoch 411/500\n",
      "56/56 [==============================] - 0s 91us/step - loss: 0.2491 - acc: 0.8750\n",
      "Epoch 412/500\n",
      "56/56 [==============================] - 0s 79us/step - loss: 0.2700 - acc: 0.8929\n",
      "Epoch 413/500\n",
      "56/56 [==============================] - 0s 96us/step - loss: 0.2317 - acc: 0.9286\n",
      "Epoch 414/500\n",
      "56/56 [==============================] - 0s 75us/step - loss: 0.1897 - acc: 0.9643\n",
      "Epoch 415/500\n",
      "56/56 [==============================] - 0s 75us/step - loss: 0.2421 - acc: 0.8214\n",
      "Epoch 416/500\n",
      "56/56 [==============================] - 0s 85us/step - loss: 0.3100 - acc: 0.8393\n",
      "Epoch 417/500\n",
      "56/56 [==============================] - 0s 74us/step - loss: 0.2015 - acc: 0.9107\n",
      "Epoch 418/500\n",
      "56/56 [==============================] - 0s 78us/step - loss: 0.2529 - acc: 0.8571\n",
      "Epoch 419/500\n",
      "56/56 [==============================] - 0s 106us/step - loss: 0.3382 - acc: 0.8036\n",
      "Epoch 420/500\n",
      "56/56 [==============================] - 0s 81us/step - loss: 0.1866 - acc: 0.8929\n",
      "Epoch 421/500\n",
      "56/56 [==============================] - 0s 96us/step - loss: 0.4363 - acc: 0.7857\n",
      "Epoch 422/500\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "56/56 [==============================] - 0s 99us/step - loss: 0.2788 - acc: 0.8393\n",
      "Epoch 423/500\n",
      "56/56 [==============================] - 0s 82us/step - loss: 0.2705 - acc: 0.8750\n",
      "Epoch 424/500\n",
      "56/56 [==============================] - 0s 78us/step - loss: 0.2407 - acc: 0.8571\n",
      "Epoch 425/500\n",
      "56/56 [==============================] - 0s 91us/step - loss: 0.2025 - acc: 0.9107\n",
      "Epoch 426/500\n",
      "56/56 [==============================] - 0s 78us/step - loss: 0.2208 - acc: 0.9286\n",
      "Epoch 427/500\n",
      "56/56 [==============================] - 0s 78us/step - loss: 0.2895 - acc: 0.8393\n",
      "Epoch 428/500\n",
      "56/56 [==============================] - 0s 84us/step - loss: 0.2260 - acc: 0.8750\n",
      "Epoch 429/500\n",
      "56/56 [==============================] - 0s 81us/step - loss: 0.2327 - acc: 0.8929\n",
      "Epoch 430/500\n",
      "56/56 [==============================] - 0s 83us/step - loss: 0.2579 - acc: 0.8750\n",
      "Epoch 431/500\n",
      "56/56 [==============================] - 0s 104us/step - loss: 0.1970 - acc: 0.9643\n",
      "Epoch 432/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.1758 - acc: 0.9464\n",
      "Epoch 433/500\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.2299 - acc: 0.8929\n",
      "Epoch 434/500\n",
      "56/56 [==============================] - 0s 75us/step - loss: 0.2580 - acc: 0.8571\n",
      "Epoch 435/500\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.2748 - acc: 0.8571\n",
      "Epoch 436/500\n",
      "56/56 [==============================] - 0s 67us/step - loss: 0.2504 - acc: 0.8929\n",
      "Epoch 437/500\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.2497 - acc: 0.8571\n",
      "Epoch 438/500\n",
      "56/56 [==============================] - 0s 106us/step - loss: 0.2053 - acc: 0.8929\n",
      "Epoch 439/500\n",
      "56/56 [==============================] - 0s 63us/step - loss: 0.2756 - acc: 0.8571\n",
      "Epoch 440/500\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.2126 - acc: 0.9107\n",
      "Epoch 441/500\n",
      "56/56 [==============================] - 0s 84us/step - loss: 0.2074 - acc: 0.9464\n",
      "Epoch 442/500\n",
      "56/56 [==============================] - 0s 172us/step - loss: 0.2734 - acc: 0.8393\n",
      "Epoch 443/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.2020 - acc: 0.9286\n",
      "Epoch 444/500\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.2842 - acc: 0.8929\n",
      "Epoch 445/500\n",
      "56/56 [==============================] - 0s 64us/step - loss: 0.2636 - acc: 0.8750\n",
      "Epoch 446/500\n",
      "56/56 [==============================] - 0s 78us/step - loss: 0.2291 - acc: 0.8571\n",
      "Epoch 447/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.2531 - acc: 0.8571\n",
      "Epoch 448/500\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.1889 - acc: 0.8929\n",
      "Epoch 449/500\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.2202 - acc: 0.9107\n",
      "Epoch 450/500\n",
      "56/56 [==============================] - 0s 58us/step - loss: 0.2991 - acc: 0.8393\n",
      "Epoch 451/500\n",
      "56/56 [==============================] - 0s 86us/step - loss: 0.2754 - acc: 0.9107\n",
      "Epoch 452/500\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.2126 - acc: 0.8929\n",
      "Epoch 453/500\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.2151 - acc: 0.8929\n",
      "Epoch 454/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.2761 - acc: 0.8750\n",
      "Epoch 455/500\n",
      "56/56 [==============================] - 0s 74us/step - loss: 0.2676 - acc: 0.8571\n",
      "Epoch 456/500\n",
      "56/56 [==============================] - 0s 64us/step - loss: 0.2657 - acc: 0.8750\n",
      "Epoch 457/500\n",
      "56/56 [==============================] - 0s 63us/step - loss: 0.2354 - acc: 0.8750\n",
      "Epoch 458/500\n",
      "56/56 [==============================] - 0s 91us/step - loss: 0.2334 - acc: 0.8214\n",
      "Epoch 459/500\n",
      "56/56 [==============================] - 0s 86us/step - loss: 0.2090 - acc: 0.8929\n",
      "Epoch 460/500\n",
      "56/56 [==============================] - 0s 65us/step - loss: 0.1975 - acc: 0.8750\n",
      "Epoch 461/500\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.2363 - acc: 0.8393\n",
      "Epoch 462/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.2013 - acc: 0.8929\n",
      "Epoch 463/500\n",
      "56/56 [==============================] - 0s 61us/step - loss: 0.2652 - acc: 0.8036\n",
      "Epoch 464/500\n",
      "56/56 [==============================] - 0s 61us/step - loss: 0.2596 - acc: 0.8214\n",
      "Epoch 465/500\n",
      "56/56 [==============================] - 0s 80us/step - loss: 0.2299 - acc: 0.8929\n",
      "Epoch 466/500\n",
      "56/56 [==============================] - 0s 80us/step - loss: 0.2467 - acc: 0.8571\n",
      "Epoch 467/500\n",
      "56/56 [==============================] - 0s 77us/step - loss: 0.1958 - acc: 0.9107\n",
      "Epoch 468/500\n",
      "56/56 [==============================] - 0s 61us/step - loss: 0.2022 - acc: 0.9107\n",
      "Epoch 469/500\n",
      "56/56 [==============================] - 0s 99us/step - loss: 0.2498 - acc: 0.8571\n",
      "Epoch 470/500\n",
      "56/56 [==============================] - 0s 64us/step - loss: 0.1826 - acc: 0.8929\n",
      "Epoch 471/500\n",
      "56/56 [==============================] - 0s 67us/step - loss: 0.2517 - acc: 0.8393\n",
      "Epoch 472/500\n",
      "56/56 [==============================] - 0s 60us/step - loss: 0.2128 - acc: 0.9107\n",
      "Epoch 473/500\n",
      "56/56 [==============================] - 0s 75us/step - loss: 0.2268 - acc: 0.8750\n",
      "Epoch 474/500\n",
      "56/56 [==============================] - 0s 74us/step - loss: 0.1905 - acc: 0.9286\n",
      "Epoch 475/500\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.2116 - acc: 0.8571\n",
      "Epoch 476/500\n",
      "56/56 [==============================] - 0s 76us/step - loss: 0.1538 - acc: 0.9643\n",
      "Epoch 477/500\n",
      "56/56 [==============================] - 0s 75us/step - loss: 0.2304 - acc: 0.8571\n",
      "Epoch 478/500\n",
      "56/56 [==============================] - 0s 62us/step - loss: 0.1982 - acc: 0.8929\n",
      "Epoch 479/500\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.2000 - acc: 0.8929\n",
      "Epoch 480/500\n",
      "56/56 [==============================] - 0s 85us/step - loss: 0.2103 - acc: 0.8929\n",
      "Epoch 481/500\n",
      "56/56 [==============================] - 0s 80us/step - loss: 0.1807 - acc: 0.9107\n",
      "Epoch 482/500\n",
      "56/56 [==============================] - 0s 73us/step - loss: 0.2520 - acc: 0.9107\n",
      "Epoch 483/500\n",
      "56/56 [==============================] - 0s 84us/step - loss: 0.2282 - acc: 0.8750\n",
      "Epoch 484/500\n",
      "56/56 [==============================] - 0s 85us/step - loss: 0.1835 - acc: 0.9286\n",
      "Epoch 485/500\n",
      "56/56 [==============================] - 0s 69us/step - loss: 0.2758 - acc: 0.8929\n",
      "Epoch 486/500\n",
      "56/56 [==============================] - 0s 64us/step - loss: 0.2305 - acc: 0.9107\n",
      "Epoch 487/500\n",
      "56/56 [==============================] - 0s 76us/step - loss: 0.1681 - acc: 0.9107\n",
      "Epoch 488/500\n",
      "56/56 [==============================] - 0s 77us/step - loss: 0.2828 - acc: 0.8750\n",
      "Epoch 489/500\n",
      "56/56 [==============================] - 0s 64us/step - loss: 0.3038 - acc: 0.8750\n",
      "Epoch 490/500\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.3866 - acc: 0.8036\n",
      "Epoch 491/500\n",
      "56/56 [==============================] - 0s 70us/step - loss: 0.2756 - acc: 0.8214\n",
      "Epoch 492/500\n",
      "56/56 [==============================] - 0s 60us/step - loss: 0.2007 - acc: 0.9107\n",
      "Epoch 493/500\n",
      "56/56 [==============================] - 0s 62us/step - loss: 0.1829 - acc: 0.8929\n",
      "Epoch 494/500\n",
      "56/56 [==============================] - 0s 57us/step - loss: 0.2896 - acc: 0.8571\n",
      "Epoch 495/500\n",
      "56/56 [==============================] - 0s 68us/step - loss: 0.2584 - acc: 0.8393\n",
      "Epoch 496/500\n",
      "56/56 [==============================] - 0s 83us/step - loss: 0.2933 - acc: 0.8571\n",
      "Epoch 497/500\n",
      "56/56 [==============================] - 0s 66us/step - loss: 0.1988 - acc: 0.9107\n",
      "Epoch 498/500\n",
      "56/56 [==============================] - 0s 76us/step - loss: 0.2371 - acc: 0.8750\n",
      "Epoch 499/500\n",
      "56/56 [==============================] - 0s 72us/step - loss: 0.1893 - acc: 0.8929\n",
      "Epoch 500/500\n",
      "56/56 [==============================] - 0s 71us/step - loss: 0.1702 - acc: 0.8929\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7fde801d40b8>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "opt = SGD(lr=0.01)\n",
    "model = Sequential()\n",
    "model.add(Dense(64, activation='relu', input_shape=(50,)))\n",
    "model.add(Dropout(0.5))\n",
    "model.add(Dense(64, activation='relu'))\n",
    "model.add(Dropout(0.5))\n",
    "model.add(Dense(64, activation='relu'))\n",
    "model.add(Dropout(0.5))\n",
    "model.add(Dense(y.shape[1], activation='softmax'))\n",
    "model.compile(optimizer='Adam',\n",
    "              loss='categorical_crossentropy',\n",
    "              metrics=['accuracy'])\n",
    "model.fit(X, y, epochs=500,batch_size=25)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Adding More Data and Beginning Data Augmentation\n",
    "\n",
    "We've started to improve upon our inital approach. Our accuracy is now approaching 0.76. But now, let's add more examples, and switch between a `train` and `test` dataset.\n",
    "\n",
    "I've run the data collection program again, and this time doubled the examples we have to train on. Now that we better understand our data, we can speed up it's import and cleaning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
      " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
      " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
      " 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.\n",
      " 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.\n",
      " 1. 1. 1. 1. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2.\n",
      " 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2.\n",
      " 2. 2. 2.]\n",
      "171 total new samples\n"
     ]
    }
   ],
   "source": [
    "dabDataset = np.load('data/more-dabs.npy')\n",
    "tposeDataset = np.load('data/more-tposes.npy')\n",
    "otherDataset = np.load('data/more-other.npy')\n",
    "labels1 = np.zeros(len(otherDataset))\n",
    "labels1 = np.append(labels1, np.full((len(dabDataset)), 1))\n",
    "labels1 = np.append(labels1, np.full((len(tposeDataset)), 2))\n",
    "print(labels1)\n",
    "print(\"%i total new samples\" % len(labels1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(171, 25, 3)\n",
      "(171, 25, 2)\n",
      "(171, 50)\n"
     ]
    }
   ],
   "source": [
    "dataset1 = np.append(otherDataset, dabDataset, axis=0)\n",
    "dataset1 = np.append(dataset1, tposeDataset, axis=0)\n",
    "X1, y1 = shuffle(dataset1, labels1)\n",
    "y1 = to_categorical(y1, 3)\n",
    "print(X1.shape)\n",
    "X1[:,:,0] = X1[:,:,0] / 720 # I think the dimensions are 1280 x 720 ?\n",
    "X1[:,:,1] = X1[:,:,1] / 1280  # let's see?\n",
    "X1 = X1[:,:,1:]\n",
    "print(X1.shape)\n",
    "X1 = X1.reshape(len(X1), 50)      # we got rid of confidence percentage\n",
    "print(X1.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/2000\n",
      "171/171 [==============================] - 0s 1ms/step - loss: 1.1371 - acc: 0.4094\n",
      "Epoch 2/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 1.0732 - acc: 0.4795\n",
      "Epoch 3/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 1.1044 - acc: 0.3626\n",
      "Epoch 4/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 1.0715 - acc: 0.3860\n",
      "Epoch 5/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.9873 - acc: 0.5263\n",
      "Epoch 6/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.9624 - acc: 0.5322\n",
      "Epoch 7/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.9664 - acc: 0.5029\n",
      "Epoch 8/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.9284 - acc: 0.5848\n",
      "Epoch 9/2000\n",
      "171/171 [==============================] - 0s 38us/step - loss: 0.8881 - acc: 0.5731\n",
      "Epoch 10/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.8896 - acc: 0.5731\n",
      "Epoch 11/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.8169 - acc: 0.6842\n",
      "Epoch 12/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.8130 - acc: 0.6667\n",
      "Epoch 13/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.7389 - acc: 0.6959\n",
      "Epoch 14/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.7201 - acc: 0.7018\n",
      "Epoch 15/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.7248 - acc: 0.7251\n",
      "Epoch 16/2000\n",
      "171/171 [==============================] - 0s 63us/step - loss: 0.6527 - acc: 0.7310\n",
      "Epoch 17/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.6444 - acc: 0.7544\n",
      "Epoch 18/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.6190 - acc: 0.7193\n",
      "Epoch 19/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.5889 - acc: 0.7544\n",
      "Epoch 20/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.6330 - acc: 0.7193\n",
      "Epoch 21/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.6211 - acc: 0.7485\n",
      "Epoch 22/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.5920 - acc: 0.7251\n",
      "Epoch 23/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.5359 - acc: 0.8070\n",
      "Epoch 24/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.5524 - acc: 0.7895\n",
      "Epoch 25/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.4928 - acc: 0.8129\n",
      "Epoch 26/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.5196 - acc: 0.7836\n",
      "Epoch 27/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.4998 - acc: 0.8480\n",
      "Epoch 28/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.5198 - acc: 0.8070\n",
      "Epoch 29/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.4595 - acc: 0.8187\n",
      "Epoch 30/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.4464 - acc: 0.8596\n",
      "Epoch 31/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.4135 - acc: 0.8596\n",
      "Epoch 32/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.4532 - acc: 0.8246\n",
      "Epoch 33/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.4044 - acc: 0.8421\n",
      "Epoch 34/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.4188 - acc: 0.8363\n",
      "Epoch 35/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.4013 - acc: 0.8129\n",
      "Epoch 36/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.4298 - acc: 0.8363\n",
      "Epoch 37/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.3770 - acc: 0.8713\n",
      "Epoch 38/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.3568 - acc: 0.8596\n",
      "Epoch 39/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.3762 - acc: 0.8538\n",
      "Epoch 40/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.3255 - acc: 0.8947\n",
      "Epoch 41/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.3799 - acc: 0.8713\n",
      "Epoch 42/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.3443 - acc: 0.8889\n",
      "Epoch 43/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.3142 - acc: 0.8772\n",
      "Epoch 44/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.3075 - acc: 0.8830\n",
      "Epoch 45/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.3339 - acc: 0.8713\n",
      "Epoch 46/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.2961 - acc: 0.9123\n",
      "Epoch 47/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.2889 - acc: 0.9064\n",
      "Epoch 48/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.2800 - acc: 0.9006\n",
      "Epoch 49/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.2636 - acc: 0.9181\n",
      "Epoch 50/2000\n",
      "171/171 [==============================] - 0s 72us/step - loss: 0.2902 - acc: 0.9006\n",
      "Epoch 51/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.2643 - acc: 0.8947\n",
      "Epoch 52/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.2746 - acc: 0.8947\n",
      "Epoch 53/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.2511 - acc: 0.9123\n",
      "Epoch 54/2000\n",
      "171/171 [==============================] - 0s 39us/step - loss: 0.2546 - acc: 0.9064\n",
      "Epoch 55/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.2626 - acc: 0.9006\n",
      "Epoch 56/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.2516 - acc: 0.9006\n",
      "Epoch 57/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.2429 - acc: 0.9298\n",
      "Epoch 58/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.2000 - acc: 0.9474\n",
      "Epoch 59/2000\n",
      "171/171 [==============================] - 0s 38us/step - loss: 0.2087 - acc: 0.9240\n",
      "Epoch 60/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.2604 - acc: 0.9006\n",
      "Epoch 61/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.2158 - acc: 0.9474\n",
      "Epoch 62/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.2317 - acc: 0.9123\n",
      "Epoch 63/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.2108 - acc: 0.9240\n",
      "Epoch 64/2000\n",
      "171/171 [==============================] - 0s 70us/step - loss: 0.2001 - acc: 0.9357\n",
      "Epoch 65/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.2004 - acc: 0.9298\n",
      "Epoch 66/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.2108 - acc: 0.9357\n",
      "Epoch 67/2000\n",
      "171/171 [==============================] - 0s 70us/step - loss: 0.1896 - acc: 0.9357\n",
      "Epoch 68/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.2183 - acc: 0.9181\n",
      "Epoch 69/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.2030 - acc: 0.9357\n",
      "Epoch 70/2000\n",
      "171/171 [==============================] - 0s 39us/step - loss: 0.2154 - acc: 0.9240\n",
      "Epoch 71/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.2256 - acc: 0.9006\n",
      "Epoch 72/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.1578 - acc: 0.9649\n",
      "Epoch 73/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.1692 - acc: 0.9532\n",
      "Epoch 74/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.1729 - acc: 0.9532\n",
      "Epoch 75/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.1479 - acc: 0.9474\n",
      "Epoch 76/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.1832 - acc: 0.9357\n",
      "Epoch 77/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.1783 - acc: 0.9123\n",
      "Epoch 78/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.1499 - acc: 0.9415\n",
      "Epoch 79/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.1577 - acc: 0.9298\n",
      "Epoch 80/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.1301 - acc: 0.9708\n",
      "Epoch 81/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.1692 - acc: 0.9064\n",
      "Epoch 82/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.1411 - acc: 0.9649\n",
      "Epoch 83/2000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "171/171 [==============================] - 0s 45us/step - loss: 0.1441 - acc: 0.9591\n",
      "Epoch 84/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.1269 - acc: 0.9591\n",
      "Epoch 85/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.1232 - acc: 0.9766\n",
      "Epoch 86/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.1393 - acc: 0.9649\n",
      "Epoch 87/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.1428 - acc: 0.9415\n",
      "Epoch 88/2000\n",
      "171/171 [==============================] - 0s 60us/step - loss: 0.1361 - acc: 0.9591\n",
      "Epoch 89/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.1016 - acc: 0.9766\n",
      "Epoch 90/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.1265 - acc: 0.9591\n",
      "Epoch 91/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.1230 - acc: 0.9474\n",
      "Epoch 92/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.1240 - acc: 0.9532\n",
      "Epoch 93/2000\n",
      "171/171 [==============================] - 0s 62us/step - loss: 0.1178 - acc: 0.9591\n",
      "Epoch 94/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.1485 - acc: 0.9357\n",
      "Epoch 95/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0962 - acc: 0.9825\n",
      "Epoch 96/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.1192 - acc: 0.9532\n",
      "Epoch 97/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0995 - acc: 0.9591\n",
      "Epoch 98/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0797 - acc: 0.9766\n",
      "Epoch 99/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.1320 - acc: 0.9474\n",
      "Epoch 100/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0697 - acc: 0.9942\n",
      "Epoch 101/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0927 - acc: 0.9708\n",
      "Epoch 102/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.1012 - acc: 0.9708\n",
      "Epoch 103/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0963 - acc: 0.9708\n",
      "Epoch 104/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0724 - acc: 0.9825\n",
      "Epoch 105/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0698 - acc: 0.9942\n",
      "Epoch 106/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.1000 - acc: 0.9415\n",
      "Epoch 107/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0981 - acc: 0.9766\n",
      "Epoch 108/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0701 - acc: 1.0000\n",
      "Epoch 109/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0689 - acc: 0.9825\n",
      "Epoch 110/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0827 - acc: 0.9649\n",
      "Epoch 111/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0765 - acc: 0.9883\n",
      "Epoch 112/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0908 - acc: 0.9532\n",
      "Epoch 113/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0684 - acc: 0.9766\n",
      "Epoch 114/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0856 - acc: 0.9708\n",
      "Epoch 115/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0695 - acc: 0.9883\n",
      "Epoch 116/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0627 - acc: 0.9942\n",
      "Epoch 117/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.1016 - acc: 0.9532\n",
      "Epoch 118/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0869 - acc: 0.9708\n",
      "Epoch 119/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0742 - acc: 0.9766\n",
      "Epoch 120/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0946 - acc: 0.9825\n",
      "Epoch 121/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0963 - acc: 0.9766\n",
      "Epoch 122/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0621 - acc: 0.9883\n",
      "Epoch 123/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0661 - acc: 0.9766\n",
      "Epoch 124/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0509 - acc: 0.9942\n",
      "Epoch 125/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0840 - acc: 0.9708\n",
      "Epoch 126/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0808 - acc: 0.9825\n",
      "Epoch 127/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0540 - acc: 1.0000\n",
      "Epoch 128/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0473 - acc: 1.0000\n",
      "Epoch 129/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0469 - acc: 0.9883\n",
      "Epoch 130/2000\n",
      "171/171 [==============================] - 0s 60us/step - loss: 0.0783 - acc: 0.9766\n",
      "Epoch 131/2000\n",
      "171/171 [==============================] - 0s 60us/step - loss: 0.0775 - acc: 0.9766\n",
      "Epoch 132/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0729 - acc: 0.9649\n",
      "Epoch 133/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0564 - acc: 0.9883\n",
      "Epoch 134/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0490 - acc: 0.9825\n",
      "Epoch 135/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0383 - acc: 0.9942\n",
      "Epoch 136/2000\n",
      "171/171 [==============================] - 0s 64us/step - loss: 0.0586 - acc: 0.9708\n",
      "Epoch 137/2000\n",
      "171/171 [==============================] - 0s 39us/step - loss: 0.0520 - acc: 0.9825\n",
      "Epoch 138/2000\n",
      "171/171 [==============================] - 0s 61us/step - loss: 0.0899 - acc: 0.9766\n",
      "Epoch 139/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0552 - acc: 0.9766\n",
      "Epoch 140/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0888 - acc: 0.9649\n",
      "Epoch 141/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0511 - acc: 0.9883\n",
      "Epoch 142/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0408 - acc: 0.9942\n",
      "Epoch 143/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0362 - acc: 0.9942\n",
      "Epoch 144/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0571 - acc: 0.9942\n",
      "Epoch 145/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0461 - acc: 0.9825\n",
      "Epoch 146/2000\n",
      "171/171 [==============================] - 0s 38us/step - loss: 0.0518 - acc: 0.9883\n",
      "Epoch 147/2000\n",
      "171/171 [==============================] - 0s 68us/step - loss: 0.0334 - acc: 1.0000\n",
      "Epoch 148/2000\n",
      "171/171 [==============================] - 0s 74us/step - loss: 0.0655 - acc: 0.9825\n",
      "Epoch 149/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0255 - acc: 1.0000\n",
      "Epoch 150/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0423 - acc: 0.9942\n",
      "Epoch 151/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0563 - acc: 0.9825\n",
      "Epoch 152/2000\n",
      "171/171 [==============================] - 0s 63us/step - loss: 0.0358 - acc: 0.9883\n",
      "Epoch 153/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0446 - acc: 0.9883\n",
      "Epoch 154/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0423 - acc: 0.9883\n",
      "Epoch 155/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0476 - acc: 0.9883\n",
      "Epoch 156/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0312 - acc: 1.0000\n",
      "Epoch 157/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0464 - acc: 0.9825\n",
      "Epoch 158/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0311 - acc: 0.9883\n",
      "Epoch 159/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0418 - acc: 0.9883\n",
      "Epoch 160/2000\n",
      "171/171 [==============================] - 0s 60us/step - loss: 0.0392 - acc: 0.9883\n",
      "Epoch 161/2000\n",
      "171/171 [==============================] - 0s 64us/step - loss: 0.0269 - acc: 0.9942\n",
      "Epoch 162/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0314 - acc: 0.9883\n",
      "Epoch 163/2000\n",
      "171/171 [==============================] - ETA: 0s - loss: 0.0155 - acc: 1.000 - 0s 57us/step - loss: 0.0408 - acc: 0.9883\n",
      "Epoch 164/2000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "171/171 [==============================] - 0s 54us/step - loss: 0.0365 - acc: 0.9883\n",
      "Epoch 165/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0265 - acc: 1.0000\n",
      "Epoch 166/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0430 - acc: 0.9883\n",
      "Epoch 167/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0535 - acc: 0.9766\n",
      "Epoch 168/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0433 - acc: 0.9883\n",
      "Epoch 169/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0281 - acc: 1.0000\n",
      "Epoch 170/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0451 - acc: 0.9942\n",
      "Epoch 171/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0600 - acc: 0.9883\n",
      "Epoch 172/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0322 - acc: 0.9942\n",
      "Epoch 173/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0446 - acc: 0.9883\n",
      "Epoch 174/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0309 - acc: 0.9942\n",
      "Epoch 175/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0307 - acc: 0.9883\n",
      "Epoch 176/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0483 - acc: 0.9883\n",
      "Epoch 177/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0437 - acc: 0.9825\n",
      "Epoch 178/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0232 - acc: 0.9942\n",
      "Epoch 179/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0154 - acc: 1.0000\n",
      "Epoch 180/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0191 - acc: 0.9942\n",
      "Epoch 181/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0358 - acc: 0.9942\n",
      "Epoch 182/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0256 - acc: 1.0000\n",
      "Epoch 183/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0576 - acc: 0.9766\n",
      "Epoch 184/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0355 - acc: 0.9942\n",
      "Epoch 185/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0257 - acc: 0.9942\n",
      "Epoch 186/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0456 - acc: 0.9825\n",
      "Epoch 187/2000\n",
      "171/171 [==============================] - 0s 68us/step - loss: 0.0526 - acc: 0.9883\n",
      "Epoch 188/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0267 - acc: 0.9942\n",
      "Epoch 189/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0246 - acc: 0.9942\n",
      "Epoch 190/2000\n",
      "171/171 [==============================] - 0s 66us/step - loss: 0.0230 - acc: 0.9942\n",
      "Epoch 191/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0339 - acc: 0.9883\n",
      "Epoch 192/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0215 - acc: 0.9942\n",
      "Epoch 193/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0363 - acc: 0.9942\n",
      "Epoch 194/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0266 - acc: 0.9942\n",
      "Epoch 195/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0343 - acc: 0.9766\n",
      "Epoch 196/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0537 - acc: 0.9825\n",
      "Epoch 197/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0301 - acc: 0.9942\n",
      "Epoch 198/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0205 - acc: 1.0000\n",
      "Epoch 199/2000\n",
      "171/171 [==============================] - 0s 63us/step - loss: 0.0453 - acc: 0.9825\n",
      "Epoch 200/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0287 - acc: 0.9942\n",
      "Epoch 201/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0336 - acc: 0.9883\n",
      "Epoch 202/2000\n",
      "171/171 [==============================] - 0s 63us/step - loss: 0.0231 - acc: 0.9942\n",
      "Epoch 203/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0202 - acc: 1.0000\n",
      "Epoch 204/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0147 - acc: 1.0000\n",
      "Epoch 205/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0314 - acc: 0.9883\n",
      "Epoch 206/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0142 - acc: 1.0000\n",
      "Epoch 207/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0310 - acc: 0.9942\n",
      "Epoch 208/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0454 - acc: 0.9883\n",
      "Epoch 209/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0151 - acc: 1.0000\n",
      "Epoch 210/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0283 - acc: 0.9942\n",
      "Epoch 211/2000\n",
      "171/171 [==============================] - 0s 61us/step - loss: 0.0225 - acc: 0.9883\n",
      "Epoch 212/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0174 - acc: 1.0000\n",
      "Epoch 213/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0198 - acc: 1.0000\n",
      "Epoch 214/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0225 - acc: 0.9942\n",
      "Epoch 215/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0199 - acc: 0.9942\n",
      "Epoch 216/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0485 - acc: 0.9825\n",
      "Epoch 217/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0330 - acc: 0.9883\n",
      "Epoch 218/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0161 - acc: 1.0000\n",
      "Epoch 219/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0160 - acc: 1.0000\n",
      "Epoch 220/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0177 - acc: 1.0000\n",
      "Epoch 221/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0116 - acc: 1.0000\n",
      "Epoch 222/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0166 - acc: 0.9942\n",
      "Epoch 223/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0302 - acc: 0.9883\n",
      "Epoch 224/2000\n",
      "171/171 [==============================] - 0s 65us/step - loss: 0.0181 - acc: 1.0000\n",
      "Epoch 225/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0349 - acc: 0.9883\n",
      "Epoch 226/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0203 - acc: 1.0000\n",
      "Epoch 227/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0411 - acc: 0.9766\n",
      "Epoch 228/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0190 - acc: 0.9942\n",
      "Epoch 229/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0634 - acc: 0.9766\n",
      "Epoch 230/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0313 - acc: 0.9942\n",
      "Epoch 231/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0307 - acc: 0.9883\n",
      "Epoch 232/2000\n",
      "171/171 [==============================] - 0s 39us/step - loss: 0.0247 - acc: 0.9942\n",
      "Epoch 233/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0297 - acc: 0.9883\n",
      "Epoch 234/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0335 - acc: 0.9883\n",
      "Epoch 235/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0252 - acc: 0.9942\n",
      "Epoch 236/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0252 - acc: 0.9942\n",
      "Epoch 237/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0339 - acc: 0.9942\n",
      "Epoch 238/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0176 - acc: 0.9942\n",
      "Epoch 239/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0153 - acc: 1.0000\n",
      "Epoch 240/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0192 - acc: 1.0000\n",
      "Epoch 241/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0097 - acc: 1.0000\n",
      "Epoch 242/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0130 - acc: 1.0000\n",
      "Epoch 243/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0147 - acc: 1.0000\n",
      "Epoch 244/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0186 - acc: 1.0000\n",
      "Epoch 245/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0376 - acc: 0.9883\n",
      "Epoch 246/2000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "171/171 [==============================] - 0s 47us/step - loss: 0.0234 - acc: 0.9942\n",
      "Epoch 247/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0293 - acc: 0.9942\n",
      "Epoch 248/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0153 - acc: 0.9942\n",
      "Epoch 249/2000\n",
      "171/171 [==============================] - 0s 39us/step - loss: 0.0572 - acc: 0.9883\n",
      "Epoch 250/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0248 - acc: 0.9883\n",
      "Epoch 251/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0170 - acc: 0.9942\n",
      "Epoch 252/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0146 - acc: 1.0000\n",
      "Epoch 253/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0169 - acc: 1.0000\n",
      "Epoch 254/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0203 - acc: 0.9942\n",
      "Epoch 255/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0109 - acc: 1.0000\n",
      "Epoch 256/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0212 - acc: 0.9942\n",
      "Epoch 257/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0133 - acc: 1.0000\n",
      "Epoch 258/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0138 - acc: 1.0000\n",
      "Epoch 259/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0089 - acc: 1.0000\n",
      "Epoch 260/2000\n",
      "171/171 [==============================] - 0s 63us/step - loss: 0.0100 - acc: 1.0000\n",
      "Epoch 261/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0097 - acc: 1.0000\n",
      "Epoch 262/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0069 - acc: 1.0000\n",
      "Epoch 263/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0131 - acc: 0.9942\n",
      "Epoch 264/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0102 - acc: 0.9942\n",
      "Epoch 265/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0224 - acc: 0.9942\n",
      "Epoch 266/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0138 - acc: 0.9942\n",
      "Epoch 267/2000\n",
      "171/171 [==============================] - 0s 63us/step - loss: 0.0130 - acc: 1.0000\n",
      "Epoch 268/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0105 - acc: 1.0000\n",
      "Epoch 269/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0209 - acc: 0.9942\n",
      "Epoch 270/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0229 - acc: 0.9942\n",
      "Epoch 271/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0230 - acc: 1.0000\n",
      "Epoch 272/2000\n",
      "171/171 [==============================] - 0s 65us/step - loss: 0.0112 - acc: 1.0000\n",
      "Epoch 273/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0445 - acc: 0.9766\n",
      "Epoch 274/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0172 - acc: 1.0000\n",
      "Epoch 275/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0166 - acc: 0.9942\n",
      "Epoch 276/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0375 - acc: 0.9766\n",
      "Epoch 277/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0064 - acc: 1.0000\n",
      "Epoch 278/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0124 - acc: 1.0000\n",
      "Epoch 279/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0209 - acc: 0.9942\n",
      "Epoch 280/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0123 - acc: 1.0000\n",
      "Epoch 281/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0105 - acc: 1.0000\n",
      "Epoch 282/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0161 - acc: 0.9942\n",
      "Epoch 283/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0040 - acc: 1.0000\n",
      "Epoch 284/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0059 - acc: 1.0000\n",
      "Epoch 285/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0055 - acc: 1.0000\n",
      "Epoch 286/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0124 - acc: 1.0000\n",
      "Epoch 287/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0134 - acc: 0.9942\n",
      "Epoch 288/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0084 - acc: 1.0000\n",
      "Epoch 289/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0107 - acc: 0.9942\n",
      "Epoch 290/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0149 - acc: 1.0000\n",
      "Epoch 291/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0068 - acc: 1.0000\n",
      "Epoch 292/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0205 - acc: 0.9883\n",
      "Epoch 293/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0181 - acc: 0.9942\n",
      "Epoch 294/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0114 - acc: 0.9942\n",
      "Epoch 295/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0119 - acc: 1.0000\n",
      "Epoch 296/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0072 - acc: 1.0000\n",
      "Epoch 297/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0178 - acc: 0.9942\n",
      "Epoch 298/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0221 - acc: 0.9942\n",
      "Epoch 299/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0200 - acc: 0.9942\n",
      "Epoch 300/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0527 - acc: 0.9825\n",
      "Epoch 301/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0307 - acc: 0.9883\n",
      "Epoch 302/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0528 - acc: 0.9825\n",
      "Epoch 303/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0274 - acc: 0.9883\n",
      "Epoch 304/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0194 - acc: 0.9942\n",
      "Epoch 305/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0236 - acc: 0.9942\n",
      "Epoch 306/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0187 - acc: 0.9883\n",
      "Epoch 307/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0100 - acc: 1.0000\n",
      "Epoch 308/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0146 - acc: 0.9942\n",
      "Epoch 309/2000\n",
      "171/171 [==============================] - ETA: 0s - loss: 0.1160 - acc: 0.960 - 0s 42us/step - loss: 0.0337 - acc: 0.9883\n",
      "Epoch 310/2000\n",
      "171/171 [==============================] - 0s 64us/step - loss: 0.0163 - acc: 0.9942\n",
      "Epoch 311/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0277 - acc: 0.9825\n",
      "Epoch 312/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0138 - acc: 1.0000\n",
      "Epoch 313/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0191 - acc: 1.0000\n",
      "Epoch 314/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0171 - acc: 0.9942\n",
      "Epoch 315/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0048 - acc: 1.0000\n",
      "Epoch 316/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0188 - acc: 0.9942\n",
      "Epoch 317/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0161 - acc: 0.9942\n",
      "Epoch 318/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0116 - acc: 1.0000\n",
      "Epoch 319/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0077 - acc: 1.0000\n",
      "Epoch 320/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0116 - acc: 0.9942\n",
      "Epoch 321/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0307 - acc: 0.9883\n",
      "Epoch 322/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0123 - acc: 1.0000\n",
      "Epoch 323/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0190 - acc: 0.9942\n",
      "Epoch 324/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0117 - acc: 1.0000\n",
      "Epoch 325/2000\n",
      "171/171 [==============================] - 0s 60us/step - loss: 0.0305 - acc: 0.9825\n",
      "Epoch 326/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0126 - acc: 0.9942\n",
      "Epoch 327/2000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "171/171 [==============================] - 0s 48us/step - loss: 0.0083 - acc: 1.0000\n",
      "Epoch 328/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0240 - acc: 0.9883\n",
      "Epoch 329/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0268 - acc: 0.9942\n",
      "Epoch 330/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0094 - acc: 0.9942\n",
      "Epoch 331/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0476 - acc: 0.9825\n",
      "Epoch 332/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0149 - acc: 0.9942\n",
      "Epoch 333/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0099 - acc: 1.0000\n",
      "Epoch 334/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0210 - acc: 0.9883\n",
      "Epoch 335/2000\n",
      "171/171 [==============================] - 0s 64us/step - loss: 0.0084 - acc: 1.0000\n",
      "Epoch 336/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0198 - acc: 0.9883\n",
      "Epoch 337/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0302 - acc: 0.9883\n",
      "Epoch 338/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0086 - acc: 1.0000\n",
      "Epoch 339/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0107 - acc: 1.0000\n",
      "Epoch 340/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0060 - acc: 1.0000\n",
      "Epoch 341/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0123 - acc: 1.0000\n",
      "Epoch 342/2000\n",
      "171/171 [==============================] - 0s 61us/step - loss: 0.0103 - acc: 1.0000\n",
      "Epoch 343/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0349 - acc: 0.9883\n",
      "Epoch 344/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0196 - acc: 0.9942\n",
      "Epoch 345/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0180 - acc: 1.0000\n",
      "Epoch 346/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0099 - acc: 1.0000\n",
      "Epoch 347/2000\n",
      "171/171 [==============================] - 0s 65us/step - loss: 0.0184 - acc: 0.9942\n",
      "Epoch 348/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0157 - acc: 0.9942\n",
      "Epoch 349/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0147 - acc: 0.9942\n",
      "Epoch 350/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0090 - acc: 1.0000\n",
      "Epoch 351/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0086 - acc: 1.0000\n",
      "Epoch 352/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0152 - acc: 0.9883\n",
      "Epoch 353/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0145 - acc: 1.0000\n",
      "Epoch 354/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0284 - acc: 0.9883\n",
      "Epoch 355/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0098 - acc: 1.0000\n",
      "Epoch 356/2000\n",
      "171/171 [==============================] - 0s 60us/step - loss: 0.0065 - acc: 1.0000\n",
      "Epoch 357/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0082 - acc: 1.0000\n",
      "Epoch 358/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0233 - acc: 0.9942\n",
      "Epoch 359/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0076 - acc: 1.0000\n",
      "Epoch 360/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0404 - acc: 0.9942\n",
      "Epoch 361/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0099 - acc: 0.9942\n",
      "Epoch 362/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0185 - acc: 0.9942\n",
      "Epoch 363/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0085 - acc: 1.0000\n",
      "Epoch 364/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0113 - acc: 1.0000\n",
      "Epoch 365/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0103 - acc: 1.0000\n",
      "Epoch 366/2000\n",
      "171/171 [==============================] - 0s 60us/step - loss: 0.0069 - acc: 1.0000\n",
      "Epoch 367/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0062 - acc: 1.0000\n",
      "Epoch 368/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0078 - acc: 1.0000\n",
      "Epoch 369/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0252 - acc: 0.9883\n",
      "Epoch 370/2000\n",
      "171/171 [==============================] - 0s 64us/step - loss: 0.0093 - acc: 1.0000\n",
      "Epoch 371/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0110 - acc: 1.0000\n",
      "Epoch 372/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0063 - acc: 1.0000\n",
      "Epoch 373/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0094 - acc: 0.9942\n",
      "Epoch 374/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0054 - acc: 1.0000\n",
      "Epoch 375/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0060 - acc: 1.0000\n",
      "Epoch 376/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0049 - acc: 1.0000\n",
      "Epoch 377/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0044 - acc: 1.0000\n",
      "Epoch 378/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0043 - acc: 1.0000\n",
      "Epoch 379/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0073 - acc: 1.0000\n",
      "Epoch 380/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0077 - acc: 1.0000\n",
      "Epoch 381/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0034 - acc: 1.0000\n",
      "Epoch 382/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0052 - acc: 1.0000\n",
      "Epoch 383/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0080 - acc: 1.0000\n",
      "Epoch 384/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0046 - acc: 1.0000\n",
      "Epoch 385/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0028 - acc: 1.0000\n",
      "Epoch 386/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0015 - acc: 1.0000\n",
      "Epoch 387/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0067 - acc: 1.0000\n",
      "Epoch 388/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0029 - acc: 1.0000\n",
      "Epoch 389/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0073 - acc: 0.9942\n",
      "Epoch 390/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0054 - acc: 1.0000\n",
      "Epoch 391/2000\n",
      "171/171 [==============================] - 0s 64us/step - loss: 0.0074 - acc: 1.0000\n",
      "Epoch 392/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0015 - acc: 1.0000\n",
      "Epoch 393/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0034 - acc: 1.0000\n",
      "Epoch 394/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0034 - acc: 1.0000\n",
      "Epoch 395/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0110 - acc: 0.9942\n",
      "Epoch 396/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0013 - acc: 1.0000\n",
      "Epoch 397/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0021 - acc: 1.0000\n",
      "Epoch 398/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0035 - acc: 1.0000\n",
      "Epoch 399/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0153 - acc: 0.9942\n",
      "Epoch 400/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0045 - acc: 1.0000\n",
      "Epoch 401/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0086 - acc: 1.0000\n",
      "Epoch 402/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0062 - acc: 1.0000\n",
      "Epoch 403/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0143 - acc: 0.9942\n",
      "Epoch 404/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0180 - acc: 0.9942\n",
      "Epoch 405/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0106 - acc: 0.9942\n",
      "Epoch 406/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0063 - acc: 1.0000\n",
      "Epoch 407/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0183 - acc: 0.9883\n",
      "Epoch 408/2000\n",
      "171/171 [==============================] - 0s 65us/step - loss: 0.0057 - acc: 1.0000\n",
      "Epoch 409/2000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "171/171 [==============================] - 0s 45us/step - loss: 0.0250 - acc: 0.9942\n",
      "Epoch 410/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0196 - acc: 0.9883\n",
      "Epoch 411/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0031 - acc: 1.0000\n",
      "Epoch 412/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0083 - acc: 1.0000\n",
      "Epoch 413/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0033 - acc: 1.0000\n",
      "Epoch 414/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0045 - acc: 1.0000\n",
      "Epoch 415/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0109 - acc: 0.9942\n",
      "Epoch 416/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0050 - acc: 1.0000\n",
      "Epoch 417/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0055 - acc: 1.0000\n",
      "Epoch 418/2000\n",
      "171/171 [==============================] - 0s 38us/step - loss: 0.0279 - acc: 0.9825\n",
      "Epoch 419/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0301 - acc: 0.9883\n",
      "Epoch 420/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0944 - acc: 0.9708\n",
      "Epoch 421/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0246 - acc: 0.9883\n",
      "Epoch 422/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.1289 - acc: 0.9298\n",
      "Epoch 423/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0640 - acc: 0.9766\n",
      "Epoch 424/2000\n",
      "171/171 [==============================] - 0s 39us/step - loss: 0.0659 - acc: 0.9766\n",
      "Epoch 425/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0722 - acc: 0.9825\n",
      "Epoch 426/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0318 - acc: 0.9883\n",
      "Epoch 427/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0082 - acc: 1.0000\n",
      "Epoch 428/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0179 - acc: 0.9942\n",
      "Epoch 429/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0118 - acc: 1.0000\n",
      "Epoch 430/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0123 - acc: 1.0000\n",
      "Epoch 431/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0225 - acc: 0.9883\n",
      "Epoch 432/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0138 - acc: 0.9942\n",
      "Epoch 433/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0066 - acc: 1.0000\n",
      "Epoch 434/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0081 - acc: 1.0000\n",
      "Epoch 435/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0069 - acc: 1.0000\n",
      "Epoch 436/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0125 - acc: 1.0000\n",
      "Epoch 437/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0131 - acc: 1.0000\n",
      "Epoch 438/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0082 - acc: 1.0000\n",
      "Epoch 439/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0045 - acc: 1.0000\n",
      "Epoch 440/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0048 - acc: 1.0000\n",
      "Epoch 441/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0236 - acc: 0.9942\n",
      "Epoch 442/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0116 - acc: 1.0000\n",
      "Epoch 443/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0104 - acc: 1.0000\n",
      "Epoch 444/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0078 - acc: 1.0000\n",
      "Epoch 445/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0154 - acc: 0.9942\n",
      "Epoch 446/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0107 - acc: 0.9942\n",
      "Epoch 447/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0124 - acc: 0.9942\n",
      "Epoch 448/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0078 - acc: 1.0000\n",
      "Epoch 449/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0097 - acc: 1.0000\n",
      "Epoch 450/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0152 - acc: 0.9942\n",
      "Epoch 451/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0033 - acc: 1.0000\n",
      "Epoch 452/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0179 - acc: 0.9942\n",
      "Epoch 453/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0114 - acc: 1.0000\n",
      "Epoch 454/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0145 - acc: 1.0000\n",
      "Epoch 455/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0104 - acc: 0.9942\n",
      "Epoch 456/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0179 - acc: 0.9883\n",
      "Epoch 457/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0278 - acc: 0.9942\n",
      "Epoch 458/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0116 - acc: 1.0000\n",
      "Epoch 459/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0221 - acc: 0.9942\n",
      "Epoch 460/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0086 - acc: 1.0000\n",
      "Epoch 461/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0056 - acc: 1.0000\n",
      "Epoch 462/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0047 - acc: 1.0000\n",
      "Epoch 463/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0047 - acc: 1.0000\n",
      "Epoch 464/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0183 - acc: 0.9942\n",
      "Epoch 465/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0047 - acc: 1.0000\n",
      "Epoch 466/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0132 - acc: 0.9942\n",
      "Epoch 467/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0228 - acc: 0.9942\n",
      "Epoch 468/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0114 - acc: 0.9942\n",
      "Epoch 469/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0235 - acc: 0.9883\n",
      "Epoch 470/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0069 - acc: 1.0000\n",
      "Epoch 471/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0098 - acc: 0.9942\n",
      "Epoch 472/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0136 - acc: 0.9942\n",
      "Epoch 473/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0242 - acc: 0.9942\n",
      "Epoch 474/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0071 - acc: 1.0000\n",
      "Epoch 475/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0154 - acc: 0.9942\n",
      "Epoch 476/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0056 - acc: 1.0000\n",
      "Epoch 477/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0096 - acc: 1.0000\n",
      "Epoch 478/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0048 - acc: 1.0000\n",
      "Epoch 479/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0163 - acc: 0.9942\n",
      "Epoch 480/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0081 - acc: 0.9942\n",
      "Epoch 481/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0057 - acc: 1.0000\n",
      "Epoch 482/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0037 - acc: 1.0000\n",
      "Epoch 483/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0220 - acc: 0.9883\n",
      "Epoch 484/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0245 - acc: 0.9942\n",
      "Epoch 485/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0161 - acc: 0.9942\n",
      "Epoch 486/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0182 - acc: 0.9942\n",
      "Epoch 487/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0031 - acc: 1.0000\n",
      "Epoch 488/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0152 - acc: 0.9942\n",
      "Epoch 489/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0061 - acc: 1.0000\n",
      "Epoch 490/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0400 - acc: 0.9883\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 491/2000\n",
      "171/171 [==============================] - 0s 63us/step - loss: 0.0121 - acc: 0.9942\n",
      "Epoch 492/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0031 - acc: 1.0000\n",
      "Epoch 493/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0069 - acc: 1.0000\n",
      "Epoch 494/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0074 - acc: 1.0000\n",
      "Epoch 495/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0150 - acc: 0.9942\n",
      "Epoch 496/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0094 - acc: 1.0000\n",
      "Epoch 497/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0199 - acc: 0.9883\n",
      "Epoch 498/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0101 - acc: 0.9942\n",
      "Epoch 499/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0095 - acc: 1.0000\n",
      "Epoch 500/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0054 - acc: 1.0000\n",
      "Epoch 501/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0098 - acc: 0.9942\n",
      "Epoch 502/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0049 - acc: 1.0000\n",
      "Epoch 503/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0075 - acc: 1.0000\n",
      "Epoch 504/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0046 - acc: 1.0000\n",
      "Epoch 505/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0040 - acc: 1.0000\n",
      "Epoch 506/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0061 - acc: 1.0000\n",
      "Epoch 507/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0033 - acc: 1.0000\n",
      "Epoch 508/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0138 - acc: 0.9942\n",
      "Epoch 509/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0058 - acc: 1.0000\n",
      "Epoch 510/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0153 - acc: 0.9883\n",
      "Epoch 511/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0054 - acc: 1.0000\n",
      "Epoch 512/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0110 - acc: 0.9942\n",
      "Epoch 513/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0150 - acc: 0.9942\n",
      "Epoch 514/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0077 - acc: 0.9942\n",
      "Epoch 515/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0020 - acc: 1.0000\n",
      "Epoch 516/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0082 - acc: 0.9942\n",
      "Epoch 517/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0084 - acc: 0.9942\n",
      "Epoch 518/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0030 - acc: 1.0000\n",
      "Epoch 519/2000\n",
      "171/171 [==============================] - 0s 65us/step - loss: 0.0015 - acc: 1.0000\n",
      "Epoch 520/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0012 - acc: 1.0000\n",
      "Epoch 521/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0033 - acc: 1.0000\n",
      "Epoch 522/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0025 - acc: 1.0000\n",
      "Epoch 523/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0039 - acc: 1.0000\n",
      "Epoch 524/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0099 - acc: 0.9942\n",
      "Epoch 525/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0027 - acc: 1.0000\n",
      "Epoch 526/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0065 - acc: 0.9942\n",
      "Epoch 527/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 528/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0105 - acc: 0.9942\n",
      "Epoch 529/2000\n",
      "171/171 [==============================] - 0s 76us/step - loss: 0.0112 - acc: 0.9942\n",
      "Epoch 530/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0062 - acc: 1.0000\n",
      "Epoch 531/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0085 - acc: 0.9942\n",
      "Epoch 532/2000\n",
      "171/171 [==============================] - 0s 60us/step - loss: 0.0054 - acc: 1.0000\n",
      "Epoch 533/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0246 - acc: 0.9883\n",
      "Epoch 534/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0047 - acc: 1.0000\n",
      "Epoch 535/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0145 - acc: 0.9942\n",
      "Epoch 536/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0104 - acc: 0.9942\n",
      "Epoch 537/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0089 - acc: 1.0000\n",
      "Epoch 538/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0022 - acc: 1.0000\n",
      "Epoch 539/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0095 - acc: 0.9942\n",
      "Epoch 540/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0201 - acc: 0.9883\n",
      "Epoch 541/2000\n",
      "171/171 [==============================] - 0s 78us/step - loss: 0.0093 - acc: 0.9942\n",
      "Epoch 542/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0027 - acc: 1.0000\n",
      "Epoch 543/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0149 - acc: 0.9942\n",
      "Epoch 544/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0146 - acc: 0.9942\n",
      "Epoch 545/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0168 - acc: 0.9942\n",
      "Epoch 546/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0183 - acc: 0.9883\n",
      "Epoch 547/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0090 - acc: 1.0000\n",
      "Epoch 548/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0118 - acc: 0.9942\n",
      "Epoch 549/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0308 - acc: 0.9883\n",
      "Epoch 550/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0095 - acc: 0.9942\n",
      "Epoch 551/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0211 - acc: 0.9942\n",
      "Epoch 552/2000\n",
      "171/171 [==============================] - 0s 78us/step - loss: 0.0050 - acc: 1.0000\n",
      "Epoch 553/2000\n",
      "171/171 [==============================] - 0s 65us/step - loss: 0.0148 - acc: 0.9942\n",
      "Epoch 554/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0059 - acc: 1.0000\n",
      "Epoch 555/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0135 - acc: 0.9942\n",
      "Epoch 556/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0053 - acc: 1.0000\n",
      "Epoch 557/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0323 - acc: 0.9825\n",
      "Epoch 558/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0061 - acc: 1.0000\n",
      "Epoch 559/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0267 - acc: 0.9942\n",
      "Epoch 560/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0462 - acc: 0.9825\n",
      "Epoch 561/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0239 - acc: 0.9942\n",
      "Epoch 562/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0155 - acc: 0.9883\n",
      "Epoch 563/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0497 - acc: 0.9883\n",
      "Epoch 564/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0094 - acc: 1.0000\n",
      "Epoch 565/2000\n",
      "171/171 [==============================] - 0s 61us/step - loss: 0.0285 - acc: 0.9883\n",
      "Epoch 566/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0107 - acc: 0.9942\n",
      "Epoch 567/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0617 - acc: 0.9649\n",
      "Epoch 568/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0272 - acc: 0.9883\n",
      "Epoch 569/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0370 - acc: 0.9825\n",
      "Epoch 570/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0505 - acc: 0.9883\n",
      "Epoch 571/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0148 - acc: 0.9942\n",
      "Epoch 572/2000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "171/171 [==============================] - 0s 48us/step - loss: 0.0089 - acc: 1.0000\n",
      "Epoch 573/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0123 - acc: 0.9942\n",
      "Epoch 574/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0142 - acc: 0.9942\n",
      "Epoch 575/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0043 - acc: 1.0000\n",
      "Epoch 576/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0059 - acc: 1.0000\n",
      "Epoch 577/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0069 - acc: 1.0000\n",
      "Epoch 578/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0019 - acc: 1.0000\n",
      "Epoch 579/2000\n",
      "171/171 [==============================] - 0s 60us/step - loss: 0.0028 - acc: 1.0000\n",
      "Epoch 580/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0253 - acc: 0.9942\n",
      "Epoch 581/2000\n",
      "171/171 [==============================] - 0s 62us/step - loss: 0.0047 - acc: 1.0000\n",
      "Epoch 582/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0116 - acc: 0.9942\n",
      "Epoch 583/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0061 - acc: 1.0000\n",
      "Epoch 584/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0071 - acc: 1.0000\n",
      "Epoch 585/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0222 - acc: 0.9942\n",
      "Epoch 586/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0119 - acc: 0.9942\n",
      "Epoch 587/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0080 - acc: 1.0000\n",
      "Epoch 588/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0171 - acc: 0.9942\n",
      "Epoch 589/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0047 - acc: 1.0000\n",
      "Epoch 590/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0026 - acc: 1.0000\n",
      "Epoch 591/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0045 - acc: 1.0000\n",
      "Epoch 592/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0048 - acc: 1.0000\n",
      "Epoch 593/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0057 - acc: 1.0000\n",
      "Epoch 594/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0092 - acc: 1.0000\n",
      "Epoch 595/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0231 - acc: 0.9942\n",
      "Epoch 596/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0043 - acc: 1.0000\n",
      "Epoch 597/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0048 - acc: 1.0000\n",
      "Epoch 598/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0295 - acc: 0.9883\n",
      "Epoch 599/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0061 - acc: 1.0000\n",
      "Epoch 600/2000\n",
      "171/171 [==============================] - 0s 63us/step - loss: 0.0130 - acc: 0.9942\n",
      "Epoch 601/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0035 - acc: 1.0000\n",
      "Epoch 602/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0073 - acc: 1.0000\n",
      "Epoch 603/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0073 - acc: 0.9942\n",
      "Epoch 604/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0067 - acc: 1.0000\n",
      "Epoch 605/2000\n",
      "171/171 [==============================] - 0s 65us/step - loss: 0.0077 - acc: 1.0000\n",
      "Epoch 606/2000\n",
      "171/171 [==============================] - 0s 66us/step - loss: 0.0153 - acc: 0.9942\n",
      "Epoch 607/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0028 - acc: 1.0000\n",
      "Epoch 608/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0107 - acc: 0.9942\n",
      "Epoch 609/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0238 - acc: 0.9883\n",
      "Epoch 610/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0018 - acc: 1.0000\n",
      "Epoch 611/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0414 - acc: 0.9942\n",
      "Epoch 612/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0102 - acc: 1.0000\n",
      "Epoch 613/2000\n",
      "171/171 [==============================] - 0s 63us/step - loss: 0.0022 - acc: 1.0000\n",
      "Epoch 614/2000\n",
      "171/171 [==============================] - 0s 61us/step - loss: 0.0030 - acc: 1.0000\n",
      "Epoch 615/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0257 - acc: 0.9883\n",
      "Epoch 616/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0151 - acc: 0.9942\n",
      "Epoch 617/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0087 - acc: 1.0000\n",
      "Epoch 618/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0074 - acc: 0.9942\n",
      "Epoch 619/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0100 - acc: 0.9942\n",
      "Epoch 620/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0021 - acc: 1.0000\n",
      "Epoch 621/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0074 - acc: 0.9942\n",
      "Epoch 622/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0027 - acc: 1.0000\n",
      "Epoch 623/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0062 - acc: 1.0000\n",
      "Epoch 624/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0067 - acc: 1.0000\n",
      "Epoch 625/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0114 - acc: 0.9942\n",
      "Epoch 626/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0061 - acc: 1.0000\n",
      "Epoch 627/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0054 - acc: 1.0000\n",
      "Epoch 628/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0026 - acc: 1.0000\n",
      "Epoch 629/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0061 - acc: 1.0000\n",
      "Epoch 630/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0125 - acc: 0.9942\n",
      "Epoch 631/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0025 - acc: 1.0000\n",
      "Epoch 632/2000\n",
      "171/171 [==============================] - 0s 39us/step - loss: 0.0092 - acc: 0.9942\n",
      "Epoch 633/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0022 - acc: 1.0000\n",
      "Epoch 634/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0053 - acc: 1.0000\n",
      "Epoch 635/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0013 - acc: 1.0000\n",
      "Epoch 636/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0079 - acc: 0.9942\n",
      "Epoch 637/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0121 - acc: 0.9942\n",
      "Epoch 638/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 9.8246e-04 - acc: 1.0000\n",
      "Epoch 639/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0019 - acc: 1.0000\n",
      "Epoch 640/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0103 - acc: 0.9942\n",
      "Epoch 641/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0093 - acc: 0.9942\n",
      "Epoch 642/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0245 - acc: 0.9883\n",
      "Epoch 643/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0178 - acc: 0.9942\n",
      "Epoch 644/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0348 - acc: 0.9825\n",
      "Epoch 645/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0050 - acc: 1.0000\n",
      "Epoch 646/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0071 - acc: 1.0000\n",
      "Epoch 647/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0036 - acc: 1.0000\n",
      "Epoch 648/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0085 - acc: 1.0000\n",
      "Epoch 649/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0070 - acc: 1.0000\n",
      "Epoch 650/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 651/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0058 - acc: 1.0000\n",
      "Epoch 652/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0015 - acc: 1.0000\n",
      "Epoch 653/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0042 - acc: 1.0000\n",
      "Epoch 654/2000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "171/171 [==============================] - 0s 45us/step - loss: 0.0026 - acc: 1.0000\n",
      "Epoch 655/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 656/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0015 - acc: 1.0000\n",
      "Epoch 657/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0084 - acc: 0.9942\n",
      "Epoch 658/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0237 - acc: 0.9942\n",
      "Epoch 659/2000\n",
      "171/171 [==============================] - 0s 77us/step - loss: 8.2528e-04 - acc: 1.0000\n",
      "Epoch 660/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0023 - acc: 1.0000\n",
      "Epoch 661/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0087 - acc: 0.9942\n",
      "Epoch 662/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0064 - acc: 1.0000\n",
      "Epoch 663/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0026 - acc: 1.0000\n",
      "Epoch 664/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0020 - acc: 1.0000\n",
      "Epoch 665/2000\n",
      "171/171 [==============================] - 0s 65us/step - loss: 0.0031 - acc: 1.0000\n",
      "Epoch 666/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0032 - acc: 1.0000\n",
      "Epoch 667/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0017 - acc: 1.0000\n",
      "Epoch 668/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 669/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0024 - acc: 1.0000\n",
      "Epoch 670/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0036 - acc: 1.0000\n",
      "Epoch 671/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0025 - acc: 1.0000\n",
      "Epoch 672/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0050 - acc: 1.0000\n",
      "Epoch 673/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0029 - acc: 1.0000\n",
      "Epoch 674/2000\n",
      "171/171 [==============================] - 0s 68us/step - loss: 0.0067 - acc: 1.0000\n",
      "Epoch 675/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 7.2328e-04 - acc: 1.0000\n",
      "Epoch 676/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0069 - acc: 1.0000\n",
      "Epoch 677/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 7.1642e-04 - acc: 1.0000\n",
      "Epoch 678/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 679/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0027 - acc: 1.0000\n",
      "Epoch 680/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 9.4010e-04 - acc: 1.0000\n",
      "Epoch 681/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0010 - acc: 1.0000\n",
      "Epoch 682/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0015 - acc: 1.0000\n",
      "Epoch 683/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0057 - acc: 0.9942\n",
      "Epoch 684/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0013 - acc: 1.0000\n",
      "Epoch 685/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0018 - acc: 1.0000\n",
      "Epoch 686/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 687/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0044 - acc: 1.0000\n",
      "Epoch 688/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 8.3329e-04 - acc: 1.0000\n",
      "Epoch 689/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0017 - acc: 1.0000\n",
      "Epoch 690/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0020 - acc: 1.0000\n",
      "Epoch 691/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0028 - acc: 1.0000\n",
      "Epoch 692/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 4.8402e-04 - acc: 1.0000\n",
      "Epoch 693/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0034 - acc: 1.0000\n",
      "Epoch 694/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0049 - acc: 0.9942\n",
      "Epoch 695/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0010 - acc: 1.0000\n",
      "Epoch 696/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 9.9021e-04 - acc: 1.0000\n",
      "Epoch 697/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0049 - acc: 1.0000\n",
      "Epoch 698/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0025 - acc: 1.0000\n",
      "Epoch 699/2000\n",
      "171/171 [==============================] - 0s 62us/step - loss: 0.0060 - acc: 0.9942\n",
      "Epoch 700/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0018 - acc: 1.0000\n",
      "Epoch 701/2000\n",
      "171/171 [==============================] - 0s 61us/step - loss: 0.0046 - acc: 1.0000\n",
      "Epoch 702/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 3.0382e-04 - acc: 1.0000\n",
      "Epoch 703/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 7.6668e-04 - acc: 1.0000\n",
      "Epoch 704/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 8.6219e-04 - acc: 1.0000\n",
      "Epoch 705/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 7.0206e-04 - acc: 1.0000\n",
      "Epoch 706/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 707/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0052 - acc: 0.9942\n",
      "Epoch 708/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 4.2538e-04 - acc: 1.0000\n",
      "Epoch 709/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0028 - acc: 1.0000\n",
      "Epoch 710/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0033 - acc: 1.0000\n",
      "Epoch 711/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 4.6443e-04 - acc: 1.0000\n",
      "Epoch 712/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 2.0518e-04 - acc: 1.0000\n",
      "Epoch 713/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 9.9439e-04 - acc: 1.0000\n",
      "Epoch 714/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0039 - acc: 1.0000\n",
      "Epoch 715/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0019 - acc: 1.0000\n",
      "Epoch 716/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0012 - acc: 1.0000\n",
      "Epoch 717/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0019 - acc: 1.0000\n",
      "Epoch 718/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0015 - acc: 1.0000\n",
      "Epoch 719/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0115 - acc: 0.9942\n",
      "Epoch 720/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0032 - acc: 1.0000\n",
      "Epoch 721/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0184 - acc: 0.9942\n",
      "Epoch 722/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0077 - acc: 0.9942\n",
      "Epoch 723/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0310 - acc: 0.9942\n",
      "Epoch 724/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0260 - acc: 0.9825\n",
      "Epoch 725/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0169 - acc: 0.9942\n",
      "Epoch 726/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0212 - acc: 0.9942\n",
      "Epoch 727/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0294 - acc: 0.9883\n",
      "Epoch 728/2000\n",
      "171/171 [==============================] - 0s 39us/step - loss: 0.0154 - acc: 0.9942\n",
      "Epoch 729/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0218 - acc: 0.9883\n",
      "Epoch 730/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0037 - acc: 1.0000\n",
      "Epoch 731/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0494 - acc: 0.9825\n",
      "Epoch 732/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0382 - acc: 0.9766\n",
      "Epoch 733/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0512 - acc: 0.9825\n",
      "Epoch 734/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0694 - acc: 0.9766\n",
      "Epoch 735/2000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "171/171 [==============================] - 0s 46us/step - loss: 0.0719 - acc: 0.9708\n",
      "Epoch 736/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.1040 - acc: 0.9766\n",
      "Epoch 737/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0678 - acc: 0.9708\n",
      "Epoch 738/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0365 - acc: 0.9766\n",
      "Epoch 739/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0373 - acc: 0.9766\n",
      "Epoch 740/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.1037 - acc: 0.9708\n",
      "Epoch 741/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0152 - acc: 1.0000\n",
      "Epoch 742/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0824 - acc: 0.9708\n",
      "Epoch 743/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0355 - acc: 0.9766\n",
      "Epoch 744/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0292 - acc: 0.9883\n",
      "Epoch 745/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0306 - acc: 0.9883\n",
      "Epoch 746/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0138 - acc: 0.9942\n",
      "Epoch 747/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0047 - acc: 1.0000\n",
      "Epoch 748/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0113 - acc: 1.0000\n",
      "Epoch 749/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0199 - acc: 0.9883\n",
      "Epoch 750/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0029 - acc: 1.0000\n",
      "Epoch 751/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0090 - acc: 0.9942\n",
      "Epoch 752/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0227 - acc: 0.9883\n",
      "Epoch 753/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0102 - acc: 0.9942\n",
      "Epoch 754/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0123 - acc: 1.0000\n",
      "Epoch 755/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0141 - acc: 0.9942\n",
      "Epoch 756/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0099 - acc: 0.9942\n",
      "Epoch 757/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0022 - acc: 1.0000\n",
      "Epoch 758/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0153 - acc: 0.9942\n",
      "Epoch 759/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0031 - acc: 1.0000\n",
      "Epoch 760/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0037 - acc: 1.0000\n",
      "Epoch 761/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0042 - acc: 1.0000\n",
      "Epoch 762/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0095 - acc: 0.9942\n",
      "Epoch 763/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0097 - acc: 0.9942\n",
      "Epoch 764/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 765/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0116 - acc: 0.9942\n",
      "Epoch 766/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0019 - acc: 1.0000\n",
      "Epoch 767/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0044 - acc: 1.0000\n",
      "Epoch 768/2000\n",
      "171/171 [==============================] - 0s 61us/step - loss: 0.0393 - acc: 0.9942\n",
      "Epoch 769/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0088 - acc: 1.0000\n",
      "Epoch 770/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0048 - acc: 1.0000\n",
      "Epoch 771/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0017 - acc: 1.0000\n",
      "Epoch 772/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0098 - acc: 0.9942\n",
      "Epoch 773/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0106 - acc: 0.9942\n",
      "Epoch 774/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0018 - acc: 1.0000\n",
      "Epoch 775/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0160 - acc: 0.9942\n",
      "Epoch 776/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0037 - acc: 1.0000\n",
      "Epoch 777/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0031 - acc: 1.0000\n",
      "Epoch 778/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0024 - acc: 1.0000\n",
      "Epoch 779/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0029 - acc: 1.0000\n",
      "Epoch 780/2000\n",
      "171/171 [==============================] - 0s 61us/step - loss: 0.0050 - acc: 1.0000\n",
      "Epoch 781/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0022 - acc: 1.0000\n",
      "Epoch 782/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0012 - acc: 1.0000\n",
      "Epoch 783/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0021 - acc: 1.0000\n",
      "Epoch 784/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0123 - acc: 0.9942\n",
      "Epoch 785/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0048 - acc: 1.0000\n",
      "Epoch 786/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0027 - acc: 1.0000\n",
      "Epoch 787/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0024 - acc: 1.0000\n",
      "Epoch 788/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 789/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 790/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0028 - acc: 1.0000\n",
      "Epoch 791/2000\n",
      "171/171 [==============================] - 0s 67us/step - loss: 0.0037 - acc: 1.0000\n",
      "Epoch 792/2000\n",
      "171/171 [==============================] - 0s 66us/step - loss: 0.0036 - acc: 1.0000\n",
      "Epoch 793/2000\n",
      "171/171 [==============================] - 0s 60us/step - loss: 8.4766e-04 - acc: 1.0000\n",
      "Epoch 794/2000\n",
      "171/171 [==============================] - 0s 63us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 795/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 796/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0027 - acc: 1.0000\n",
      "Epoch 797/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0042 - acc: 1.0000\n",
      "Epoch 798/2000\n",
      "171/171 [==============================] - 0s 69us/step - loss: 5.5148e-04 - acc: 1.0000\n",
      "Epoch 799/2000\n",
      "171/171 [==============================] - 0s 66us/step - loss: 0.0205 - acc: 0.9883\n",
      "Epoch 800/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0124 - acc: 0.9942\n",
      "Epoch 801/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0054 - acc: 1.0000\n",
      "Epoch 802/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0034 - acc: 1.0000\n",
      "Epoch 803/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0116 - acc: 0.9942\n",
      "Epoch 804/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0070 - acc: 0.9942\n",
      "Epoch 805/2000\n",
      "171/171 [==============================] - 0s 66us/step - loss: 0.0021 - acc: 1.0000\n",
      "Epoch 806/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0032 - acc: 1.0000\n",
      "Epoch 807/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0013 - acc: 1.0000\n",
      "Epoch 808/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0051 - acc: 0.9942\n",
      "Epoch 809/2000\n",
      "171/171 [==============================] - 0s 63us/step - loss: 0.0029 - acc: 1.0000\n",
      "Epoch 810/2000\n",
      "171/171 [==============================] - 0s 72us/step - loss: 0.0023 - acc: 1.0000\n",
      "Epoch 811/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0074 - acc: 0.9942\n",
      "Epoch 812/2000\n",
      "171/171 [==============================] - 0s 66us/step - loss: 7.7662e-04 - acc: 1.0000\n",
      "Epoch 813/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0266 - acc: 0.9942\n",
      "Epoch 814/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0136 - acc: 0.9942\n",
      "Epoch 815/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0101 - acc: 0.9942\n",
      "Epoch 816/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0030 - acc: 1.0000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 817/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0025 - acc: 1.0000\n",
      "Epoch 818/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 9.4336e-04 - acc: 1.0000\n",
      "Epoch 819/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0029 - acc: 1.0000\n",
      "Epoch 820/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0025 - acc: 1.0000\n",
      "Epoch 821/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0148 - acc: 0.9942\n",
      "Epoch 822/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0027 - acc: 1.0000\n",
      "Epoch 823/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0055 - acc: 1.0000\n",
      "Epoch 824/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 825/2000\n",
      "171/171 [==============================] - 0s 62us/step - loss: 0.0024 - acc: 1.0000\n",
      "Epoch 826/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0027 - acc: 1.0000\n",
      "Epoch 827/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0040 - acc: 1.0000\n",
      "Epoch 828/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0044 - acc: 1.0000\n",
      "Epoch 829/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0064 - acc: 1.0000\n",
      "Epoch 830/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0020 - acc: 1.0000\n",
      "Epoch 831/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0028 - acc: 1.0000\n",
      "Epoch 832/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0018 - acc: 1.0000\n",
      "Epoch 833/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 3.4020e-04 - acc: 1.0000\n",
      "Epoch 834/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0047 - acc: 1.0000\n",
      "Epoch 835/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0022 - acc: 1.0000\n",
      "Epoch 836/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0027 - acc: 1.0000\n",
      "Epoch 837/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0092 - acc: 0.9942\n",
      "Epoch 838/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 839/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0021 - acc: 1.0000\n",
      "Epoch 840/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0027 - acc: 1.0000\n",
      "Epoch 841/2000\n",
      "171/171 [==============================] - 0s 61us/step - loss: 0.0013 - acc: 1.0000\n",
      "Epoch 842/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0015 - acc: 1.0000\n",
      "Epoch 843/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 844/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0023 - acc: 1.0000\n",
      "Epoch 845/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0037 - acc: 1.0000\n",
      "Epoch 846/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 8.0774e-04 - acc: 1.0000\n",
      "Epoch 847/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 6.6196e-04 - acc: 1.0000\n",
      "Epoch 848/2000\n",
      "171/171 [==============================] - 0s 70us/step - loss: 0.0020 - acc: 1.0000\n",
      "Epoch 849/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0029 - acc: 1.0000\n",
      "Epoch 850/2000\n",
      "171/171 [==============================] - 0s 60us/step - loss: 4.2385e-04 - acc: 1.0000\n",
      "Epoch 851/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 852/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 3.4816e-04 - acc: 1.0000\n",
      "Epoch 853/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 3.5885e-04 - acc: 1.0000\n",
      "Epoch 854/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 6.3483e-04 - acc: 1.0000\n",
      "Epoch 855/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0021 - acc: 1.0000\n",
      "Epoch 856/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 5.9691e-04 - acc: 1.0000\n",
      "Epoch 857/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 6.4404e-04 - acc: 1.0000\n",
      "Epoch 858/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 8.3284e-04 - acc: 1.0000\n",
      "Epoch 859/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0043 - acc: 0.9942\n",
      "Epoch 860/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0071 - acc: 1.0000\n",
      "Epoch 861/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0048 - acc: 1.0000\n",
      "Epoch 862/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 5.0523e-04 - acc: 1.0000\n",
      "Epoch 863/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 864/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0151 - acc: 0.9942\n",
      "Epoch 865/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0021 - acc: 1.0000\n",
      "Epoch 866/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0050 - acc: 1.0000\n",
      "Epoch 867/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0210 - acc: 0.9942\n",
      "Epoch 868/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0232 - acc: 0.9883\n",
      "Epoch 869/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0068 - acc: 0.9942\n",
      "Epoch 870/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0047 - acc: 1.0000\n",
      "Epoch 871/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0015 - acc: 1.0000\n",
      "Epoch 872/2000\n",
      "171/171 [==============================] - 0s 65us/step - loss: 0.0045 - acc: 1.0000\n",
      "Epoch 873/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 874/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 4.9324e-04 - acc: 1.0000\n",
      "Epoch 875/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 9.5241e-04 - acc: 1.0000\n",
      "Epoch 876/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0032 - acc: 1.0000\n",
      "Epoch 877/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0018 - acc: 1.0000\n",
      "Epoch 878/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0159 - acc: 0.9942\n",
      "Epoch 879/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0085 - acc: 0.9942\n",
      "Epoch 880/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0060 - acc: 1.0000\n",
      "Epoch 881/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0036 - acc: 1.0000\n",
      "Epoch 882/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0153 - acc: 0.9942\n",
      "Epoch 883/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0114 - acc: 1.0000\n",
      "Epoch 884/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0341 - acc: 0.9825\n",
      "Epoch 885/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0104 - acc: 0.9942\n",
      "Epoch 886/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0378 - acc: 0.9825\n",
      "Epoch 887/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0232 - acc: 0.9942\n",
      "Epoch 888/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0362 - acc: 0.9942\n",
      "Epoch 889/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0169 - acc: 0.9942\n",
      "Epoch 890/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0074 - acc: 1.0000\n",
      "Epoch 891/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0176 - acc: 0.9942\n",
      "Epoch 892/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0176 - acc: 0.9942\n",
      "Epoch 893/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0269 - acc: 0.9942\n",
      "Epoch 894/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0104 - acc: 1.0000\n",
      "Epoch 895/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0481 - acc: 0.9825\n",
      "Epoch 896/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0050 - acc: 1.0000\n",
      "Epoch 897/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0064 - acc: 1.0000\n",
      "Epoch 898/2000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "171/171 [==============================] - 0s 49us/step - loss: 0.0074 - acc: 1.0000\n",
      "Epoch 899/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0076 - acc: 1.0000\n",
      "Epoch 900/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0041 - acc: 1.0000\n",
      "Epoch 901/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0068 - acc: 1.0000\n",
      "Epoch 902/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0111 - acc: 1.0000\n",
      "Epoch 903/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 5.8147e-04 - acc: 1.0000\n",
      "Epoch 904/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0172 - acc: 0.9942\n",
      "Epoch 905/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0407 - acc: 0.9883\n",
      "Epoch 906/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0031 - acc: 1.0000\n",
      "Epoch 907/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0027 - acc: 1.0000\n",
      "Epoch 908/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0384 - acc: 0.9825\n",
      "Epoch 909/2000\n",
      "171/171 [==============================] - 0s 64us/step - loss: 0.0069 - acc: 1.0000\n",
      "Epoch 910/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0044 - acc: 1.0000\n",
      "Epoch 911/2000\n",
      "171/171 [==============================] - 0s 68us/step - loss: 0.0224 - acc: 0.9883\n",
      "Epoch 912/2000\n",
      "171/171 [==============================] - 0s 63us/step - loss: 0.0100 - acc: 0.9942\n",
      "Epoch 913/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0296 - acc: 0.9942\n",
      "Epoch 914/2000\n",
      "171/171 [==============================] - 0s 64us/step - loss: 0.0868 - acc: 0.9766\n",
      "Epoch 915/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0706 - acc: 0.9766\n",
      "Epoch 916/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0782 - acc: 0.9766\n",
      "Epoch 917/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0574 - acc: 0.9825\n",
      "Epoch 918/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0427 - acc: 0.9825\n",
      "Epoch 919/2000\n",
      "171/171 [==============================] - 0s 71us/step - loss: 0.0287 - acc: 0.9825\n",
      "Epoch 920/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0030 - acc: 1.0000\n",
      "Epoch 921/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0020 - acc: 1.0000\n",
      "Epoch 922/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0047 - acc: 1.0000\n",
      "Epoch 923/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0036 - acc: 1.0000\n",
      "Epoch 924/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0084 - acc: 1.0000\n",
      "Epoch 925/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0074 - acc: 1.0000\n",
      "Epoch 926/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 927/2000\n",
      "171/171 [==============================] - 0s 62us/step - loss: 0.0052 - acc: 1.0000\n",
      "Epoch 928/2000\n",
      "171/171 [==============================] - 0s 60us/step - loss: 0.0137 - acc: 0.9942\n",
      "Epoch 929/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0029 - acc: 1.0000\n",
      "Epoch 930/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0023 - acc: 1.0000\n",
      "Epoch 931/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0193 - acc: 0.9883\n",
      "Epoch 932/2000\n",
      "171/171 [==============================] - 0s 63us/step - loss: 8.8381e-04 - acc: 1.0000\n",
      "Epoch 933/2000\n",
      "171/171 [==============================] - 0s 63us/step - loss: 0.0019 - acc: 1.0000\n",
      "Epoch 934/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0198 - acc: 0.9942\n",
      "Epoch 935/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0028 - acc: 1.0000\n",
      "Epoch 936/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0041 - acc: 1.0000\n",
      "Epoch 937/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0026 - acc: 1.0000\n",
      "Epoch 938/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0057 - acc: 0.9942\n",
      "Epoch 939/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0032 - acc: 1.0000\n",
      "Epoch 940/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0074 - acc: 1.0000\n",
      "Epoch 941/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0120 - acc: 0.9942\n",
      "Epoch 942/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0086 - acc: 0.9942\n",
      "Epoch 943/2000\n",
      "171/171 [==============================] - 0s 60us/step - loss: 0.0068 - acc: 1.0000\n",
      "Epoch 944/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0038 - acc: 1.0000\n",
      "Epoch 945/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0043 - acc: 1.0000\n",
      "Epoch 946/2000\n",
      "171/171 [==============================] - 0s 76us/step - loss: 0.0027 - acc: 1.0000\n",
      "Epoch 947/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0032 - acc: 1.0000\n",
      "Epoch 948/2000\n",
      "171/171 [==============================] - 0s 99us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 949/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0096 - acc: 0.9942\n",
      "Epoch 950/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0081 - acc: 0.9942\n",
      "Epoch 951/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0035 - acc: 1.0000\n",
      "Epoch 952/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0028 - acc: 1.0000\n",
      "Epoch 953/2000\n",
      "171/171 [==============================] - 0s 66us/step - loss: 0.0022 - acc: 1.0000\n",
      "Epoch 954/2000\n",
      "171/171 [==============================] - 0s 61us/step - loss: 0.0029 - acc: 1.0000\n",
      "Epoch 955/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0041 - acc: 1.0000\n",
      "Epoch 956/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0013 - acc: 1.0000\n",
      "Epoch 957/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 958/2000\n",
      "171/171 [==============================] - 0s 62us/step - loss: 0.0023 - acc: 1.0000\n",
      "Epoch 959/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0017 - acc: 1.0000\n",
      "Epoch 960/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0015 - acc: 1.0000\n",
      "Epoch 961/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 7.2916e-04 - acc: 1.0000\n",
      "Epoch 962/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0012 - acc: 1.0000\n",
      "Epoch 963/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 964/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0030 - acc: 1.0000\n",
      "Epoch 965/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 9.0618e-04 - acc: 1.0000\n",
      "Epoch 966/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0044 - acc: 1.0000\n",
      "Epoch 967/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0017 - acc: 1.0000\n",
      "Epoch 968/2000\n",
      "171/171 [==============================] - 0s 66us/step - loss: 0.0070 - acc: 0.9942\n",
      "Epoch 969/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0026 - acc: 1.0000\n",
      "Epoch 970/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0040 - acc: 1.0000\n",
      "Epoch 971/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0018 - acc: 1.0000\n",
      "Epoch 972/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0012 - acc: 1.0000\n",
      "Epoch 973/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0015 - acc: 1.0000\n",
      "Epoch 974/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 975/2000\n",
      "171/171 [==============================] - 0s 78us/step - loss: 7.3859e-04 - acc: 1.0000\n",
      "Epoch 976/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0028 - acc: 1.0000\n",
      "Epoch 977/2000\n",
      "171/171 [==============================] - 0s 60us/step - loss: 8.5100e-04 - acc: 1.0000\n",
      "Epoch 978/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0013 - acc: 1.0000\n",
      "Epoch 979/2000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "171/171 [==============================] - 0s 56us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 980/2000\n",
      "171/171 [==============================] - 0s 60us/step - loss: 8.6044e-04 - acc: 1.0000\n",
      "Epoch 981/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0152 - acc: 0.9942\n",
      "Epoch 982/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0327 - acc: 0.9883\n",
      "Epoch 983/2000\n",
      "171/171 [==============================] - 0s 62us/step - loss: 0.0025 - acc: 1.0000\n",
      "Epoch 984/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0068 - acc: 1.0000\n",
      "Epoch 985/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0053 - acc: 1.0000\n",
      "Epoch 986/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0010 - acc: 1.0000\n",
      "Epoch 987/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0059 - acc: 1.0000\n",
      "Epoch 988/2000\n",
      "171/171 [==============================] - 0s 64us/step - loss: 0.0017 - acc: 1.0000\n",
      "Epoch 989/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0027 - acc: 1.0000\n",
      "Epoch 990/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0023 - acc: 1.0000\n",
      "Epoch 991/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0051 - acc: 1.0000\n",
      "Epoch 992/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0021 - acc: 1.0000\n",
      "Epoch 993/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 3.5617e-04 - acc: 1.0000\n",
      "Epoch 994/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0031 - acc: 1.0000\n",
      "Epoch 995/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0024 - acc: 1.0000\n",
      "Epoch 996/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0103 - acc: 0.9942\n",
      "Epoch 997/2000\n",
      "171/171 [==============================] - 0s 60us/step - loss: 0.0015 - acc: 1.0000\n",
      "Epoch 998/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0038 - acc: 1.0000\n",
      "Epoch 999/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0074 - acc: 1.0000\n",
      "Epoch 1000/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0023 - acc: 1.0000\n",
      "Epoch 1001/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0035 - acc: 1.0000\n",
      "Epoch 1002/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0162 - acc: 0.9942\n",
      "Epoch 1003/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0035 - acc: 1.0000\n",
      "Epoch 1004/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0219 - acc: 0.9942\n",
      "Epoch 1005/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0066 - acc: 1.0000\n",
      "Epoch 1006/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0039 - acc: 1.0000\n",
      "Epoch 1007/2000\n",
      "171/171 [==============================] - 0s 63us/step - loss: 0.0072 - acc: 1.0000\n",
      "Epoch 1008/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0053 - acc: 1.0000\n",
      "Epoch 1009/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0025 - acc: 1.0000\n",
      "Epoch 1010/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0120 - acc: 0.9942\n",
      "Epoch 1011/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0030 - acc: 1.0000\n",
      "Epoch 1012/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0043 - acc: 1.0000\n",
      "Epoch 1013/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0025 - acc: 1.0000\n",
      "Epoch 1014/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0075 - acc: 1.0000\n",
      "Epoch 1015/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0368 - acc: 0.9883\n",
      "Epoch 1016/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0174 - acc: 0.9942\n",
      "Epoch 1017/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0447 - acc: 0.9883\n",
      "Epoch 1018/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0906 - acc: 0.9883\n",
      "Epoch 1019/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0550 - acc: 0.9766\n",
      "Epoch 1020/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0285 - acc: 0.9942\n",
      "Epoch 1021/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0065 - acc: 1.0000\n",
      "Epoch 1022/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0099 - acc: 0.9942\n",
      "Epoch 1023/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0076 - acc: 0.9942\n",
      "Epoch 1024/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0041 - acc: 1.0000\n",
      "Epoch 1025/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0017 - acc: 1.0000\n",
      "Epoch 1026/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0028 - acc: 1.0000\n",
      "Epoch 1027/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0328 - acc: 0.9942\n",
      "Epoch 1028/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0027 - acc: 1.0000\n",
      "Epoch 1029/2000\n",
      "171/171 [==============================] - 0s 67us/step - loss: 0.0174 - acc: 0.9942\n",
      "Epoch 1030/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0394 - acc: 0.9825\n",
      "Epoch 1031/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0256 - acc: 0.9942\n",
      "Epoch 1032/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0113 - acc: 0.9942\n",
      "Epoch 1033/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0203 - acc: 0.9942\n",
      "Epoch 1034/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0050 - acc: 1.0000\n",
      "Epoch 1035/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0024 - acc: 1.0000\n",
      "Epoch 1036/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0121 - acc: 0.9942\n",
      "Epoch 1037/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0268 - acc: 0.9883\n",
      "Epoch 1038/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 1039/2000\n",
      "171/171 [==============================] - 0s 85us/step - loss: 0.0024 - acc: 1.0000\n",
      "Epoch 1040/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0185 - acc: 0.9883\n",
      "Epoch 1041/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0062 - acc: 1.0000\n",
      "Epoch 1042/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0123 - acc: 0.9942\n",
      "Epoch 1043/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0037 - acc: 1.0000\n",
      "Epoch 1044/2000\n",
      "171/171 [==============================] - 0s 73us/step - loss: 0.0229 - acc: 0.9942\n",
      "Epoch 1045/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0087 - acc: 0.9942\n",
      "Epoch 1046/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0020 - acc: 1.0000\n",
      "Epoch 1047/2000\n",
      "171/171 [==============================] - 0s 62us/step - loss: 0.0159 - acc: 0.9883\n",
      "Epoch 1048/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0203 - acc: 0.9942\n",
      "Epoch 1049/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0178 - acc: 0.9942\n",
      "Epoch 1050/2000\n",
      "171/171 [==============================] - 0s 60us/step - loss: 0.0509 - acc: 0.9825\n",
      "Epoch 1051/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0481 - acc: 0.9766\n",
      "Epoch 1052/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0164 - acc: 0.9942\n",
      "Epoch 1053/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0057 - acc: 1.0000\n",
      "Epoch 1054/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0054 - acc: 1.0000\n",
      "Epoch 1055/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0144 - acc: 0.9942\n",
      "Epoch 1056/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0112 - acc: 0.9942\n",
      "Epoch 1057/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0074 - acc: 0.9942\n",
      "Epoch 1058/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0076 - acc: 0.9942\n",
      "Epoch 1059/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0267 - acc: 0.9883\n",
      "Epoch 1060/2000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "171/171 [==============================] - 0s 46us/step - loss: 0.0075 - acc: 1.0000\n",
      "Epoch 1061/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0049 - acc: 1.0000\n",
      "Epoch 1062/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0022 - acc: 1.0000\n",
      "Epoch 1063/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0069 - acc: 1.0000\n",
      "Epoch 1064/2000\n",
      "171/171 [==============================] - 0s 61us/step - loss: 0.0027 - acc: 1.0000\n",
      "Epoch 1065/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0066 - acc: 1.0000\n",
      "Epoch 1066/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0179 - acc: 0.9942\n",
      "Epoch 1067/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0018 - acc: 1.0000\n",
      "Epoch 1068/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0041 - acc: 1.0000\n",
      "Epoch 1069/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0171 - acc: 0.9942\n",
      "Epoch 1070/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 1071/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0034 - acc: 1.0000\n",
      "Epoch 1072/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0041 - acc: 1.0000\n",
      "Epoch 1073/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0050 - acc: 1.0000\n",
      "Epoch 1074/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0025 - acc: 1.0000\n",
      "Epoch 1075/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0024 - acc: 1.0000\n",
      "Epoch 1076/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0058 - acc: 1.0000\n",
      "Epoch 1077/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0013 - acc: 1.0000\n",
      "Epoch 1078/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0020 - acc: 1.0000\n",
      "Epoch 1079/2000\n",
      "171/171 [==============================] - 0s 65us/step - loss: 9.7067e-04 - acc: 1.0000\n",
      "Epoch 1080/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0013 - acc: 1.0000\n",
      "Epoch 1081/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 7.0516e-04 - acc: 1.0000\n",
      "Epoch 1082/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0013 - acc: 1.0000\n",
      "Epoch 1083/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0019 - acc: 1.0000\n",
      "Epoch 1084/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 6.1288e-04 - acc: 1.0000\n",
      "Epoch 1085/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 7.7176e-04 - acc: 1.0000\n",
      "Epoch 1086/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 9.8621e-04 - acc: 1.0000\n",
      "Epoch 1087/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0010 - acc: 1.0000\n",
      "Epoch 1088/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0031 - acc: 1.0000\n",
      "Epoch 1089/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 2.9385e-04 - acc: 1.0000\n",
      "Epoch 1090/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 8.8706e-04 - acc: 1.0000\n",
      "Epoch 1091/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 6.4789e-04 - acc: 1.0000\n",
      "Epoch 1092/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0035 - acc: 1.0000\n",
      "Epoch 1093/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 7.3243e-04 - acc: 1.0000\n",
      "Epoch 1094/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0075 - acc: 1.0000\n",
      "Epoch 1095/2000\n",
      "171/171 [==============================] - 0s 63us/step - loss: 0.0023 - acc: 1.0000\n",
      "Epoch 1096/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 6.3140e-04 - acc: 1.0000\n",
      "Epoch 1097/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 1098/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 6.9015e-04 - acc: 1.0000\n",
      "Epoch 1099/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 4.1840e-04 - acc: 1.0000\n",
      "Epoch 1100/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0044 - acc: 1.0000\n",
      "Epoch 1101/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 6.8378e-04 - acc: 1.0000\n",
      "Epoch 1102/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0036 - acc: 1.0000\n",
      "Epoch 1103/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0030 - acc: 1.0000\n",
      "Epoch 1104/2000\n",
      "171/171 [==============================] - 0s 61us/step - loss: 0.0119 - acc: 0.9942\n",
      "Epoch 1105/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0051 - acc: 0.9942\n",
      "Epoch 1106/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0325 - acc: 0.9942\n",
      "Epoch 1107/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0019 - acc: 1.0000\n",
      "Epoch 1108/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0098 - acc: 0.9942\n",
      "Epoch 1109/2000\n",
      "171/171 [==============================] - 0s 66us/step - loss: 0.0040 - acc: 1.0000\n",
      "Epoch 1110/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0230 - acc: 0.9942\n",
      "Epoch 1111/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0031 - acc: 1.0000\n",
      "Epoch 1112/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0034 - acc: 1.0000\n",
      "Epoch 1113/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0017 - acc: 1.0000\n",
      "Epoch 1114/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0015 - acc: 1.0000\n",
      "Epoch 1115/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0012 - acc: 1.0000\n",
      "Epoch 1116/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0058 - acc: 1.0000\n",
      "Epoch 1117/2000\n",
      "171/171 [==============================] - 0s 66us/step - loss: 0.0042 - acc: 1.0000\n",
      "Epoch 1118/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0179 - acc: 0.9942\n",
      "Epoch 1119/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0039 - acc: 1.0000\n",
      "Epoch 1120/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0023 - acc: 1.0000\n",
      "Epoch 1121/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0022 - acc: 1.0000\n",
      "Epoch 1122/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 4.3712e-04 - acc: 1.0000\n",
      "Epoch 1123/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0020 - acc: 1.0000\n",
      "Epoch 1124/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 1125/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0010 - acc: 1.0000\n",
      "Epoch 1126/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 6.4997e-04 - acc: 1.0000\n",
      "Epoch 1127/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0048 - acc: 1.0000\n",
      "Epoch 1128/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 6.6027e-04 - acc: 1.0000\n",
      "Epoch 1129/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0013 - acc: 1.0000\n",
      "Epoch 1130/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 4.5153e-04 - acc: 1.0000\n",
      "Epoch 1131/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0025 - acc: 1.0000\n",
      "Epoch 1132/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0046 - acc: 1.0000\n",
      "Epoch 1133/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0019 - acc: 1.0000\n",
      "Epoch 1134/2000\n",
      "171/171 [==============================] - 0s 39us/step - loss: 6.9170e-04 - acc: 1.0000\n",
      "Epoch 1135/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 6.8258e-04 - acc: 1.0000\n",
      "Epoch 1136/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0033 - acc: 1.0000\n",
      "Epoch 1137/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0015 - acc: 1.0000\n",
      "Epoch 1138/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0124 - acc: 0.9942\n",
      "Epoch 1139/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 1140/2000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "171/171 [==============================] - 0s 39us/step - loss: 2.4169e-04 - acc: 1.0000\n",
      "Epoch 1141/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0020 - acc: 1.0000\n",
      "Epoch 1142/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0019 - acc: 1.0000\n",
      "Epoch 1143/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0019 - acc: 1.0000\n",
      "Epoch 1144/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 8.1213e-04 - acc: 1.0000\n",
      "Epoch 1145/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0032 - acc: 1.0000\n",
      "Epoch 1146/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0031 - acc: 1.0000\n",
      "Epoch 1147/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0036 - acc: 1.0000\n",
      "Epoch 1148/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0019 - acc: 1.0000\n",
      "Epoch 1149/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 7.4878e-04 - acc: 1.0000\n",
      "Epoch 1150/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 5.3130e-04 - acc: 1.0000\n",
      "Epoch 1151/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 5.0042e-04 - acc: 1.0000\n",
      "Epoch 1152/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 4.1964e-04 - acc: 1.0000\n",
      "Epoch 1153/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 2.5912e-04 - acc: 1.0000\n",
      "Epoch 1154/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 6.3624e-04 - acc: 1.0000\n",
      "Epoch 1155/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 1.6914e-04 - acc: 1.0000\n",
      "Epoch 1156/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0057 - acc: 0.9942\n",
      "Epoch 1157/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 6.4515e-04 - acc: 1.0000\n",
      "Epoch 1158/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0299 - acc: 0.9942\n",
      "Epoch 1159/2000\n",
      "171/171 [==============================] - 0s 61us/step - loss: 8.0640e-04 - acc: 1.0000\n",
      "Epoch 1160/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0034 - acc: 1.0000\n",
      "Epoch 1161/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0038 - acc: 1.0000\n",
      "Epoch 1162/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0029 - acc: 1.0000\n",
      "Epoch 1163/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 8.0498e-04 - acc: 1.0000\n",
      "Epoch 1164/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 4.5616e-04 - acc: 1.0000\n",
      "Epoch 1165/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0062 - acc: 0.9942\n",
      "Epoch 1166/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0140 - acc: 0.9942\n",
      "Epoch 1167/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0218 - acc: 0.9883\n",
      "Epoch 1168/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 1169/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 7.4869e-04 - acc: 1.0000\n",
      "Epoch 1170/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0084 - acc: 0.9942\n",
      "Epoch 1171/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0088 - acc: 0.9942\n",
      "Epoch 1172/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0082 - acc: 0.9942\n",
      "Epoch 1173/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0111 - acc: 0.9942\n",
      "Epoch 1174/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 1175/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0242 - acc: 0.9883\n",
      "Epoch 1176/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0052 - acc: 1.0000\n",
      "Epoch 1177/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0123 - acc: 0.9942\n",
      "Epoch 1178/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0051 - acc: 1.0000\n",
      "Epoch 1179/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0743 - acc: 0.9825\n",
      "Epoch 1180/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0109 - acc: 0.9942\n",
      "Epoch 1181/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0027 - acc: 1.0000\n",
      "Epoch 1182/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0112 - acc: 0.9942\n",
      "Epoch 1183/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0094 - acc: 1.0000\n",
      "Epoch 1184/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0300 - acc: 0.9942\n",
      "Epoch 1185/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 5.4078e-04 - acc: 1.0000\n",
      "Epoch 1186/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0230 - acc: 0.9942\n",
      "Epoch 1187/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0163 - acc: 0.9883\n",
      "Epoch 1188/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0056 - acc: 1.0000\n",
      "Epoch 1189/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0237 - acc: 0.9942\n",
      "Epoch 1190/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0076 - acc: 1.0000\n",
      "Epoch 1191/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0024 - acc: 1.0000\n",
      "Epoch 1192/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0045 - acc: 1.0000\n",
      "Epoch 1193/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0028 - acc: 1.0000\n",
      "Epoch 1194/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0223 - acc: 0.9942\n",
      "Epoch 1195/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0133 - acc: 0.9942\n",
      "Epoch 1196/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0170 - acc: 0.9883\n",
      "Epoch 1197/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0322 - acc: 0.9942\n",
      "Epoch 1198/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0242 - acc: 0.9883\n",
      "Epoch 1199/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0371 - acc: 0.9825\n",
      "Epoch 1200/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0439 - acc: 0.9825\n",
      "Epoch 1201/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0283 - acc: 0.9883\n",
      "Epoch 1202/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0867 - acc: 0.9766\n",
      "Epoch 1203/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0373 - acc: 0.9942\n",
      "Epoch 1204/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0084 - acc: 1.0000\n",
      "Epoch 1205/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0074 - acc: 1.0000\n",
      "Epoch 1206/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0074 - acc: 1.0000\n",
      "Epoch 1207/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0037 - acc: 1.0000\n",
      "Epoch 1208/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0105 - acc: 0.9942\n",
      "Epoch 1209/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0045 - acc: 1.0000\n",
      "Epoch 1210/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0031 - acc: 1.0000\n",
      "Epoch 1211/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0017 - acc: 1.0000\n",
      "Epoch 1212/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0027 - acc: 1.0000\n",
      "Epoch 1213/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0019 - acc: 1.0000\n",
      "Epoch 1214/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 1215/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0113 - acc: 0.9942\n",
      "Epoch 1216/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0036 - acc: 1.0000\n",
      "Epoch 1217/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0021 - acc: 1.0000\n",
      "Epoch 1218/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0036 - acc: 1.0000\n",
      "Epoch 1219/2000\n",
      "171/171 [==============================] - 0s 39us/step - loss: 0.0242 - acc: 0.9942\n",
      "Epoch 1220/2000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "171/171 [==============================] - 0s 56us/step - loss: 0.0082 - acc: 1.0000\n",
      "Epoch 1221/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0188 - acc: 0.9942\n",
      "Epoch 1222/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0048 - acc: 1.0000\n",
      "Epoch 1223/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0078 - acc: 0.9942\n",
      "Epoch 1224/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0305 - acc: 0.9942\n",
      "Epoch 1225/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0047 - acc: 1.0000\n",
      "Epoch 1226/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0046 - acc: 1.0000\n",
      "Epoch 1227/2000\n",
      "171/171 [==============================] - 0s 61us/step - loss: 0.0058 - acc: 1.0000\n",
      "Epoch 1228/2000\n",
      "171/171 [==============================] - 0s 60us/step - loss: 0.0054 - acc: 0.9942\n",
      "Epoch 1229/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0120 - acc: 0.9942\n",
      "Epoch 1230/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 4.1034e-04 - acc: 1.0000\n",
      "Epoch 1231/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 1232/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0063 - acc: 1.0000\n",
      "Epoch 1233/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0095 - acc: 0.9942\n",
      "Epoch 1234/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0310 - acc: 0.9942\n",
      "Epoch 1235/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0087 - acc: 0.9942\n",
      "Epoch 1236/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0122 - acc: 0.9942\n",
      "Epoch 1237/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0017 - acc: 1.0000\n",
      "Epoch 1238/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0025 - acc: 1.0000\n",
      "Epoch 1239/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0057 - acc: 1.0000\n",
      "Epoch 1240/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0019 - acc: 1.0000\n",
      "Epoch 1241/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0125 - acc: 0.9942\n",
      "Epoch 1242/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0100 - acc: 0.9942\n",
      "Epoch 1243/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0031 - acc: 1.0000\n",
      "Epoch 1244/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0012 - acc: 1.0000\n",
      "Epoch 1245/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0182 - acc: 0.9883\n",
      "Epoch 1246/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0053 - acc: 1.0000\n",
      "Epoch 1247/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0015 - acc: 1.0000\n",
      "Epoch 1248/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0036 - acc: 1.0000\n",
      "Epoch 1249/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0047 - acc: 1.0000\n",
      "Epoch 1250/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 1251/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0046 - acc: 1.0000\n",
      "Epoch 1252/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0023 - acc: 1.0000\n",
      "Epoch 1253/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0022 - acc: 1.0000\n",
      "Epoch 1254/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 8.6969e-04 - acc: 1.0000\n",
      "Epoch 1255/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 7.5643e-04 - acc: 1.0000\n",
      "Epoch 1256/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0021 - acc: 1.0000\n",
      "Epoch 1257/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 9.0266e-04 - acc: 1.0000\n",
      "Epoch 1258/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 6.3892e-04 - acc: 1.0000\n",
      "Epoch 1259/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 2.7471e-04 - acc: 1.0000\n",
      "Epoch 1260/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 2.6295e-04 - acc: 1.0000\n",
      "Epoch 1261/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 5.3034e-04 - acc: 1.0000\n",
      "Epoch 1262/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0028 - acc: 1.0000\n",
      "Epoch 1263/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0017 - acc: 1.0000\n",
      "Epoch 1264/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 8.1638e-04 - acc: 1.0000\n",
      "Epoch 1265/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0024 - acc: 1.0000\n",
      "Epoch 1266/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 3.5033e-04 - acc: 1.0000\n",
      "Epoch 1267/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0019 - acc: 1.0000\n",
      "Epoch 1268/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 2.8673e-04 - acc: 1.0000\n",
      "Epoch 1269/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0015 - acc: 1.0000\n",
      "Epoch 1270/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0036 - acc: 1.0000\n",
      "Epoch 1271/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0065 - acc: 1.0000\n",
      "Epoch 1272/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 9.4567e-04 - acc: 1.0000\n",
      "Epoch 1273/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 5.6933e-04 - acc: 1.0000\n",
      "Epoch 1274/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0305 - acc: 0.9883\n",
      "Epoch 1275/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0023 - acc: 1.0000\n",
      "Epoch 1276/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0070 - acc: 1.0000\n",
      "Epoch 1277/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0018 - acc: 1.0000\n",
      "Epoch 1278/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0019 - acc: 1.0000\n",
      "Epoch 1279/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0176 - acc: 0.9942\n",
      "Epoch 1280/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0043 - acc: 1.0000\n",
      "Epoch 1281/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0131 - acc: 0.9883\n",
      "Epoch 1282/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 4.1269e-04 - acc: 1.0000\n",
      "Epoch 1283/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 1284/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0029 - acc: 1.0000\n",
      "Epoch 1285/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 7.3447e-04 - acc: 1.0000\n",
      "Epoch 1286/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 8.5903e-04 - acc: 1.0000\n",
      "Epoch 1287/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0040 - acc: 1.0000\n",
      "Epoch 1288/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0024 - acc: 1.0000\n",
      "Epoch 1289/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 4.0103e-04 - acc: 1.0000\n",
      "Epoch 1290/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 2.2283e-04 - acc: 1.0000\n",
      "Epoch 1291/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0035 - acc: 1.0000\n",
      "Epoch 1292/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0054 - acc: 1.0000\n",
      "Epoch 1293/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0039 - acc: 1.0000\n",
      "Epoch 1294/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0024 - acc: 1.0000\n",
      "Epoch 1295/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 8.3696e-04 - acc: 1.0000\n",
      "Epoch 1296/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0020 - acc: 1.0000\n",
      "Epoch 1297/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0255 - acc: 0.9942\n",
      "Epoch 1298/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0045 - acc: 1.0000\n",
      "Epoch 1299/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0090 - acc: 1.0000\n",
      "Epoch 1300/2000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "171/171 [==============================] - 0s 41us/step - loss: 0.0020 - acc: 1.0000\n",
      "Epoch 1301/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0010 - acc: 1.0000\n",
      "Epoch 1302/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0038 - acc: 1.0000\n",
      "Epoch 1303/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0185 - acc: 0.9942\n",
      "Epoch 1304/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 4.6773e-04 - acc: 1.0000\n",
      "Epoch 1305/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0031 - acc: 1.0000\n",
      "Epoch 1306/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 9.0796e-04 - acc: 1.0000\n",
      "Epoch 1307/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0081 - acc: 0.9942\n",
      "Epoch 1308/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 7.2273e-04 - acc: 1.0000\n",
      "Epoch 1309/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0029 - acc: 1.0000\n",
      "Epoch 1310/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0031 - acc: 1.0000\n",
      "Epoch 1311/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 1312/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0082 - acc: 0.9942\n",
      "Epoch 1313/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0020 - acc: 1.0000\n",
      "Epoch 1314/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0105 - acc: 1.0000\n",
      "Epoch 1315/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0024 - acc: 1.0000\n",
      "Epoch 1316/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0024 - acc: 1.0000\n",
      "Epoch 1317/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0019 - acc: 1.0000\n",
      "Epoch 1318/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0096 - acc: 0.9942\n",
      "Epoch 1319/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0062 - acc: 1.0000\n",
      "Epoch 1320/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0013 - acc: 1.0000\n",
      "Epoch 1321/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0026 - acc: 1.0000\n",
      "Epoch 1322/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0020 - acc: 1.0000\n",
      "Epoch 1323/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0514 - acc: 0.9883\n",
      "Epoch 1324/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0105 - acc: 0.9942\n",
      "Epoch 1325/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0083 - acc: 0.9942\n",
      "Epoch 1326/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0463 - acc: 0.9825\n",
      "Epoch 1327/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0074 - acc: 0.9942\n",
      "Epoch 1328/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0820 - acc: 0.9825\n",
      "Epoch 1329/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0709 - acc: 0.9883\n",
      "Epoch 1330/2000\n",
      "171/171 [==============================] - 0s 60us/step - loss: 0.0616 - acc: 0.9825\n",
      "Epoch 1331/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0139 - acc: 0.9942\n",
      "Epoch 1332/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0062 - acc: 1.0000\n",
      "Epoch 1333/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0179 - acc: 0.9883\n",
      "Epoch 1334/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0168 - acc: 0.9942\n",
      "Epoch 1335/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0241 - acc: 0.9942\n",
      "Epoch 1336/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0177 - acc: 0.9942\n",
      "Epoch 1337/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0135 - acc: 0.9942\n",
      "Epoch 1338/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0099 - acc: 0.9942\n",
      "Epoch 1339/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0202 - acc: 0.9942\n",
      "Epoch 1340/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0031 - acc: 1.0000\n",
      "Epoch 1341/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0092 - acc: 1.0000\n",
      "Epoch 1342/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0125 - acc: 0.9942\n",
      "Epoch 1343/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0033 - acc: 1.0000\n",
      "Epoch 1344/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0115 - acc: 0.9942\n",
      "Epoch 1345/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0345 - acc: 0.9883\n",
      "Epoch 1346/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0046 - acc: 1.0000\n",
      "Epoch 1347/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0023 - acc: 1.0000\n",
      "Epoch 1348/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0142 - acc: 0.9942\n",
      "Epoch 1349/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0027 - acc: 1.0000\n",
      "Epoch 1350/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0043 - acc: 1.0000\n",
      "Epoch 1351/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0096 - acc: 0.9942\n",
      "Epoch 1352/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0219 - acc: 0.9942\n",
      "Epoch 1353/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0235 - acc: 0.9883\n",
      "Epoch 1354/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0160 - acc: 0.9883\n",
      "Epoch 1355/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0416 - acc: 0.9825\n",
      "Epoch 1356/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0566 - acc: 0.9942\n",
      "Epoch 1357/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0248 - acc: 0.9883\n",
      "Epoch 1358/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0177 - acc: 0.9883\n",
      "Epoch 1359/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0068 - acc: 1.0000\n",
      "Epoch 1360/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0585 - acc: 0.9825\n",
      "Epoch 1361/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0043 - acc: 1.0000\n",
      "Epoch 1362/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0412 - acc: 0.9883\n",
      "Epoch 1363/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0280 - acc: 0.9825\n",
      "Epoch 1364/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0290 - acc: 0.9942\n",
      "Epoch 1365/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0071 - acc: 0.9942\n",
      "Epoch 1366/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0326 - acc: 0.9883\n",
      "Epoch 1367/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0093 - acc: 0.9942\n",
      "Epoch 1368/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0209 - acc: 0.9942\n",
      "Epoch 1369/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0318 - acc: 0.9883\n",
      "Epoch 1370/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0079 - acc: 1.0000\n",
      "Epoch 1371/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0027 - acc: 1.0000\n",
      "Epoch 1372/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0169 - acc: 0.9942\n",
      "Epoch 1373/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0304 - acc: 0.9825\n",
      "Epoch 1374/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0032 - acc: 1.0000\n",
      "Epoch 1375/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0144 - acc: 0.9942\n",
      "Epoch 1376/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0228 - acc: 0.9942\n",
      "Epoch 1377/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0018 - acc: 1.0000\n",
      "Epoch 1378/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 1379/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0018 - acc: 1.0000\n",
      "Epoch 1380/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0022 - acc: 1.0000\n",
      "Epoch 1381/2000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "171/171 [==============================] - 0s 44us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 1382/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0012 - acc: 1.0000\n",
      "Epoch 1383/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0039 - acc: 1.0000\n",
      "Epoch 1384/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 9.7058e-04 - acc: 1.0000\n",
      "Epoch 1385/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 7.5913e-04 - acc: 1.0000\n",
      "Epoch 1386/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0015 - acc: 1.0000\n",
      "Epoch 1387/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0028 - acc: 1.0000\n",
      "Epoch 1388/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0048 - acc: 1.0000\n",
      "Epoch 1389/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 1390/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0038 - acc: 1.0000\n",
      "Epoch 1391/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0025 - acc: 1.0000\n",
      "Epoch 1392/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0028 - acc: 1.0000\n",
      "Epoch 1393/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0035 - acc: 1.0000\n",
      "Epoch 1394/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 1395/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0017 - acc: 1.0000\n",
      "Epoch 1396/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0015 - acc: 1.0000\n",
      "Epoch 1397/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 7.0840e-04 - acc: 1.0000\n",
      "Epoch 1398/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0017 - acc: 1.0000\n",
      "Epoch 1399/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 5.5772e-04 - acc: 1.0000\n",
      "Epoch 1400/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0019 - acc: 1.0000\n",
      "Epoch 1401/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 5.6175e-04 - acc: 1.0000\n",
      "Epoch 1402/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0025 - acc: 1.0000\n",
      "Epoch 1403/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 4.2085e-04 - acc: 1.0000\n",
      "Epoch 1404/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0036 - acc: 1.0000\n",
      "Epoch 1405/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0028 - acc: 1.0000\n",
      "Epoch 1406/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 5.5587e-04 - acc: 1.0000\n",
      "Epoch 1407/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 4.9703e-04 - acc: 1.0000\n",
      "Epoch 1408/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 1409/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 6.0180e-04 - acc: 1.0000\n",
      "Epoch 1410/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 8.9554e-04 - acc: 1.0000\n",
      "Epoch 1411/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0121 - acc: 0.9942\n",
      "Epoch 1412/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0037 - acc: 1.0000\n",
      "Epoch 1413/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0157 - acc: 0.9883\n",
      "Epoch 1414/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 5.2810e-04 - acc: 1.0000\n",
      "Epoch 1415/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0013 - acc: 1.0000\n",
      "Epoch 1416/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0150 - acc: 0.9883\n",
      "Epoch 1417/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0015 - acc: 1.0000\n",
      "Epoch 1418/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0126 - acc: 0.9942\n",
      "Epoch 1419/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0039 - acc: 1.0000\n",
      "Epoch 1420/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0024 - acc: 1.0000\n",
      "Epoch 1421/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0021 - acc: 1.0000\n",
      "Epoch 1422/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 9.9143e-04 - acc: 1.0000\n",
      "Epoch 1423/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 5.5171e-04 - acc: 1.0000\n",
      "Epoch 1424/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 7.7590e-04 - acc: 1.0000\n",
      "Epoch 1425/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0031 - acc: 1.0000\n",
      "Epoch 1426/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 1427/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 8.1309e-04 - acc: 1.0000\n",
      "Epoch 1428/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0089 - acc: 0.9942\n",
      "Epoch 1429/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0178 - acc: 0.9942\n",
      "Epoch 1430/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0092 - acc: 0.9942\n",
      "Epoch 1431/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0017 - acc: 1.0000\n",
      "Epoch 1432/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 1433/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0124 - acc: 0.9942\n",
      "Epoch 1434/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0013 - acc: 1.0000\n",
      "Epoch 1435/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0099 - acc: 1.0000\n",
      "Epoch 1436/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0139 - acc: 0.9942\n",
      "Epoch 1437/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0326 - acc: 0.9825\n",
      "Epoch 1438/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0173 - acc: 0.9942\n",
      "Epoch 1439/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0064 - acc: 1.0000\n",
      "Epoch 1440/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0107 - acc: 0.9942\n",
      "Epoch 1441/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0230 - acc: 0.9883\n",
      "Epoch 1442/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0038 - acc: 1.0000\n",
      "Epoch 1443/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0023 - acc: 1.0000\n",
      "Epoch 1444/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0066 - acc: 0.9942\n",
      "Epoch 1445/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0248 - acc: 0.9942\n",
      "Epoch 1446/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0134 - acc: 0.9942\n",
      "Epoch 1447/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0366 - acc: 0.9942\n",
      "Epoch 1448/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 1449/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0031 - acc: 1.0000\n",
      "Epoch 1450/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0012 - acc: 1.0000\n",
      "Epoch 1451/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0022 - acc: 1.0000\n",
      "Epoch 1452/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0088 - acc: 0.9942\n",
      "Epoch 1453/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0045 - acc: 1.0000\n",
      "Epoch 1454/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0020 - acc: 1.0000\n",
      "Epoch 1455/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0021 - acc: 1.0000\n",
      "Epoch 1456/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0065 - acc: 1.0000\n",
      "Epoch 1457/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0107 - acc: 0.9942\n",
      "Epoch 1458/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0140 - acc: 0.9942\n",
      "Epoch 1459/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0065 - acc: 1.0000\n",
      "Epoch 1460/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0070 - acc: 1.0000\n",
      "Epoch 1461/2000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "171/171 [==============================] - 0s 41us/step - loss: 0.0033 - acc: 1.0000\n",
      "Epoch 1462/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 2.4371e-04 - acc: 1.0000\n",
      "Epoch 1463/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 7.2505e-04 - acc: 1.0000\n",
      "Epoch 1464/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 4.2227e-04 - acc: 1.0000\n",
      "Epoch 1465/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0021 - acc: 1.0000\n",
      "Epoch 1466/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 4.2449e-04 - acc: 1.0000\n",
      "Epoch 1467/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0037 - acc: 1.0000\n",
      "Epoch 1468/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0018 - acc: 1.0000\n",
      "Epoch 1469/2000\n",
      "171/171 [==============================] - 0s 39us/step - loss: 0.0218 - acc: 0.9883\n",
      "Epoch 1470/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0077 - acc: 1.0000\n",
      "Epoch 1471/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0024 - acc: 1.0000\n",
      "Epoch 1472/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0031 - acc: 1.0000\n",
      "Epoch 1473/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 6.3381e-04 - acc: 1.0000\n",
      "Epoch 1474/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0359 - acc: 0.9942\n",
      "Epoch 1475/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 4.4689e-04 - acc: 1.0000\n",
      "Epoch 1476/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0053 - acc: 1.0000\n",
      "Epoch 1477/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0121 - acc: 0.9942\n",
      "Epoch 1478/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0048 - acc: 1.0000\n",
      "Epoch 1479/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 7.9688e-04 - acc: 1.0000\n",
      "Epoch 1480/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0017 - acc: 1.0000\n",
      "Epoch 1481/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0025 - acc: 1.0000\n",
      "Epoch 1482/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0064 - acc: 0.9942\n",
      "Epoch 1483/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0033 - acc: 1.0000\n",
      "Epoch 1484/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0085 - acc: 0.9942\n",
      "Epoch 1485/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0057 - acc: 1.0000\n",
      "Epoch 1486/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0010 - acc: 1.0000\n",
      "Epoch 1487/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 9.8337e-04 - acc: 1.0000\n",
      "Epoch 1488/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 1489/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0062 - acc: 0.9942\n",
      "Epoch 1490/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0037 - acc: 1.0000\n",
      "Epoch 1491/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 9.6839e-04 - acc: 1.0000\n",
      "Epoch 1492/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 6.7266e-04 - acc: 1.0000\n",
      "Epoch 1493/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 6.5644e-04 - acc: 1.0000\n",
      "Epoch 1494/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0022 - acc: 1.0000\n",
      "Epoch 1495/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 7.4989e-04 - acc: 1.0000\n",
      "Epoch 1496/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 1.1446e-04 - acc: 1.0000\n",
      "Epoch 1497/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 4.8980e-04 - acc: 1.0000\n",
      "Epoch 1498/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 4.3974e-04 - acc: 1.0000\n",
      "Epoch 1499/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 4.9497e-04 - acc: 1.0000\n",
      "Epoch 1500/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 3.7924e-04 - acc: 1.0000\n",
      "Epoch 1501/2000\n",
      "171/171 [==============================] - 0s 71us/step - loss: 0.0010 - acc: 1.0000\n",
      "Epoch 1502/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 3.2220e-04 - acc: 1.0000\n",
      "Epoch 1503/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0074 - acc: 0.9942\n",
      "Epoch 1504/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 8.9893e-04 - acc: 1.0000\n",
      "Epoch 1505/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0024 - acc: 1.0000\n",
      "Epoch 1506/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 9.4986e-04 - acc: 1.0000\n",
      "Epoch 1507/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 7.6529e-04 - acc: 1.0000\n",
      "Epoch 1508/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 2.0871e-04 - acc: 1.0000\n",
      "Epoch 1509/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0197 - acc: 0.9942\n",
      "Epoch 1510/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 4.4480e-04 - acc: 1.0000\n",
      "Epoch 1511/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 4.3750e-04 - acc: 1.0000\n",
      "Epoch 1512/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0012 - acc: 1.0000\n",
      "Epoch 1513/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0031 - acc: 1.0000\n",
      "Epoch 1514/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 3.4263e-04 - acc: 1.0000\n",
      "Epoch 1515/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0134 - acc: 0.9942\n",
      "Epoch 1516/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0027 - acc: 1.0000\n",
      "Epoch 1517/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0017 - acc: 1.0000\n",
      "Epoch 1518/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0276 - acc: 0.9942\n",
      "Epoch 1519/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 1520/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0045 - acc: 0.9942\n",
      "Epoch 1521/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 8.3003e-04 - acc: 1.0000\n",
      "Epoch 1522/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0013 - acc: 1.0000\n",
      "Epoch 1523/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0034 - acc: 1.0000\n",
      "Epoch 1524/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 4.7899e-04 - acc: 1.0000\n",
      "Epoch 1525/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0118 - acc: 0.9942\n",
      "Epoch 1526/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0015 - acc: 1.0000\n",
      "Epoch 1527/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0051 - acc: 1.0000\n",
      "Epoch 1528/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 1529/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0141 - acc: 0.9942\n",
      "Epoch 1530/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0208 - acc: 0.9942\n",
      "Epoch 1531/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 9.4671e-04 - acc: 1.0000\n",
      "Epoch 1532/2000\n",
      "171/171 [==============================] - 0s 61us/step - loss: 0.0132 - acc: 0.9942\n",
      "Epoch 1533/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0058 - acc: 1.0000\n",
      "Epoch 1534/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0276 - acc: 0.9942\n",
      "Epoch 1535/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 1536/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0161 - acc: 0.9883\n",
      "Epoch 1537/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0078 - acc: 0.9942\n",
      "Epoch 1538/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0155 - acc: 0.9942\n",
      "Epoch 1539/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0062 - acc: 1.0000\n",
      "Epoch 1540/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0029 - acc: 1.0000\n",
      "Epoch 1541/2000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "171/171 [==============================] - 0s 40us/step - loss: 0.0030 - acc: 1.0000\n",
      "Epoch 1542/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0024 - acc: 1.0000\n",
      "Epoch 1543/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0039 - acc: 1.0000\n",
      "Epoch 1544/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0187 - acc: 0.9942\n",
      "Epoch 1545/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0045 - acc: 1.0000\n",
      "Epoch 1546/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0078 - acc: 1.0000\n",
      "Epoch 1547/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 3.4674e-04 - acc: 1.0000\n",
      "Epoch 1548/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0087 - acc: 0.9942\n",
      "Epoch 1549/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0389 - acc: 0.9825\n",
      "Epoch 1550/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0239 - acc: 0.9883\n",
      "Epoch 1551/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0201 - acc: 0.9883\n",
      "Epoch 1552/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0074 - acc: 0.9942\n",
      "Epoch 1553/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0025 - acc: 1.0000\n",
      "Epoch 1554/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0063 - acc: 0.9942\n",
      "Epoch 1555/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0013 - acc: 1.0000\n",
      "Epoch 1556/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0127 - acc: 0.9883\n",
      "Epoch 1557/2000\n",
      "171/171 [==============================] - 0s 66us/step - loss: 0.0129 - acc: 0.9942\n",
      "Epoch 1558/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 7.9418e-04 - acc: 1.0000\n",
      "Epoch 1559/2000\n",
      "171/171 [==============================] - 0s 70us/step - loss: 0.0096 - acc: 0.9942\n",
      "Epoch 1560/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0061 - acc: 1.0000\n",
      "Epoch 1561/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0033 - acc: 1.0000\n",
      "Epoch 1562/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 8.9293e-04 - acc: 1.0000\n",
      "Epoch 1563/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 8.1794e-04 - acc: 1.0000\n",
      "Epoch 1564/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0010 - acc: 1.0000\n",
      "Epoch 1565/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 6.1744e-04 - acc: 1.0000\n",
      "Epoch 1566/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0027 - acc: 1.0000\n",
      "Epoch 1567/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 4.6947e-04 - acc: 1.0000\n",
      "Epoch 1568/2000\n",
      "171/171 [==============================] - 0s 60us/step - loss: 5.2483e-04 - acc: 1.0000\n",
      "Epoch 1569/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0051 - acc: 1.0000\n",
      "Epoch 1570/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0185 - acc: 0.9942\n",
      "Epoch 1571/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0102 - acc: 0.9942\n",
      "Epoch 1572/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0272 - acc: 0.9883\n",
      "Epoch 1573/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0495 - acc: 0.9883\n",
      "Epoch 1574/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0332 - acc: 0.9942\n",
      "Epoch 1575/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0174 - acc: 0.9942\n",
      "Epoch 1576/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0066 - acc: 1.0000\n",
      "Epoch 1577/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0160 - acc: 0.9883\n",
      "Epoch 1578/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0027 - acc: 1.0000\n",
      "Epoch 1579/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0336 - acc: 0.9825\n",
      "Epoch 1580/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0393 - acc: 0.9708\n",
      "Epoch 1581/2000\n",
      "171/171 [==============================] - 0s 39us/step - loss: 0.0066 - acc: 1.0000\n",
      "Epoch 1582/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0147 - acc: 0.9883\n",
      "Epoch 1583/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0712 - acc: 0.9825\n",
      "Epoch 1584/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0142 - acc: 0.9942\n",
      "Epoch 1585/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0141 - acc: 0.9942\n",
      "Epoch 1586/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0136 - acc: 0.9942\n",
      "Epoch 1587/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0022 - acc: 1.0000\n",
      "Epoch 1588/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0090 - acc: 0.9942\n",
      "Epoch 1589/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0118 - acc: 0.9942\n",
      "Epoch 1590/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0076 - acc: 0.9942\n",
      "Epoch 1591/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 1592/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0033 - acc: 1.0000\n",
      "Epoch 1593/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 1594/2000\n",
      "171/171 [==============================] - 0s 62us/step - loss: 0.0013 - acc: 1.0000\n",
      "Epoch 1595/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0036 - acc: 1.0000\n",
      "Epoch 1596/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 9.7713e-04 - acc: 1.0000\n",
      "Epoch 1597/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0015 - acc: 1.0000\n",
      "Epoch 1598/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0069 - acc: 0.9942\n",
      "Epoch 1599/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0028 - acc: 1.0000\n",
      "Epoch 1600/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0233 - acc: 0.9942\n",
      "Epoch 1601/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0029 - acc: 1.0000\n",
      "Epoch 1602/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 1603/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0018 - acc: 1.0000\n",
      "Epoch 1604/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0107 - acc: 0.9942\n",
      "Epoch 1605/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0010 - acc: 1.0000\n",
      "Epoch 1606/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0113 - acc: 0.9942\n",
      "Epoch 1607/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0087 - acc: 0.9942\n",
      "Epoch 1608/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0929 - acc: 0.9883\n",
      "Epoch 1609/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0022 - acc: 1.0000\n",
      "Epoch 1610/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0162 - acc: 0.9942\n",
      "Epoch 1611/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0301 - acc: 0.9883\n",
      "Epoch 1612/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0260 - acc: 0.9883\n",
      "Epoch 1613/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0050 - acc: 1.0000\n",
      "Epoch 1614/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0088 - acc: 0.9942\n",
      "Epoch 1615/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0128 - acc: 0.9942\n",
      "Epoch 1616/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 6.2894e-04 - acc: 1.0000\n",
      "Epoch 1617/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0294 - acc: 0.9942\n",
      "Epoch 1618/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0082 - acc: 0.9942\n",
      "Epoch 1619/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0084 - acc: 1.0000\n",
      "Epoch 1620/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0142 - acc: 0.9942\n",
      "Epoch 1621/2000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "171/171 [==============================] - 0s 54us/step - loss: 0.0174 - acc: 0.9942\n",
      "Epoch 1622/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0162 - acc: 0.9942\n",
      "Epoch 1623/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0111 - acc: 0.9942\n",
      "Epoch 1624/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0026 - acc: 1.0000\n",
      "Epoch 1625/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0053 - acc: 1.0000\n",
      "Epoch 1626/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0017 - acc: 1.0000\n",
      "Epoch 1627/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 1628/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0027 - acc: 1.0000\n",
      "Epoch 1629/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 5.6839e-04 - acc: 1.0000\n",
      "Epoch 1630/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0101 - acc: 0.9942\n",
      "Epoch 1631/2000\n",
      "171/171 [==============================] - 0s 65us/step - loss: 0.0019 - acc: 1.0000\n",
      "Epoch 1632/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0018 - acc: 1.0000\n",
      "Epoch 1633/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0060 - acc: 1.0000\n",
      "Epoch 1634/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0216 - acc: 0.9883\n",
      "Epoch 1635/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 1636/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 1637/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0061 - acc: 0.9942\n",
      "Epoch 1638/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 8.2277e-04 - acc: 1.0000\n",
      "Epoch 1639/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 1640/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0044 - acc: 1.0000\n",
      "Epoch 1641/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 7.7405e-04 - acc: 1.0000\n",
      "Epoch 1642/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0026 - acc: 1.0000\n",
      "Epoch 1643/2000\n",
      "171/171 [==============================] - 0s 71us/step - loss: 2.3112e-04 - acc: 1.0000\n",
      "Epoch 1644/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 1645/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 8.7106e-04 - acc: 1.0000\n",
      "Epoch 1646/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0229 - acc: 0.9942\n",
      "Epoch 1647/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0012 - acc: 1.0000\n",
      "Epoch 1648/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0036 - acc: 1.0000\n",
      "Epoch 1649/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 2.7838e-04 - acc: 1.0000\n",
      "Epoch 1650/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0043 - acc: 1.0000\n",
      "Epoch 1651/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0039 - acc: 1.0000\n",
      "Epoch 1652/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 1653/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0021 - acc: 1.0000\n",
      "Epoch 1654/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 6.9908e-04 - acc: 1.0000\n",
      "Epoch 1655/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 6.9583e-04 - acc: 1.0000\n",
      "Epoch 1656/2000\n",
      "171/171 [==============================] - 0s 39us/step - loss: 0.0117 - acc: 0.9942\n",
      "Epoch 1657/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0043 - acc: 1.0000\n",
      "Epoch 1658/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0066 - acc: 0.9942\n",
      "Epoch 1659/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0023 - acc: 1.0000\n",
      "Epoch 1660/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0071 - acc: 0.9942\n",
      "Epoch 1661/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0047 - acc: 1.0000\n",
      "Epoch 1662/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0090 - acc: 0.9942\n",
      "Epoch 1663/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0039 - acc: 1.0000\n",
      "Epoch 1664/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0075 - acc: 0.9942\n",
      "Epoch 1665/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0125 - acc: 0.9942\n",
      "Epoch 1666/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 1.1529e-04 - acc: 1.0000\n",
      "Epoch 1667/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 4.4669e-04 - acc: 1.0000\n",
      "Epoch 1668/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0102 - acc: 0.9942\n",
      "Epoch 1669/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 1670/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0064 - acc: 1.0000\n",
      "Epoch 1671/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 7.1474e-04 - acc: 1.0000\n",
      "Epoch 1672/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 3.1273e-04 - acc: 1.0000\n",
      "Epoch 1673/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0018 - acc: 1.0000\n",
      "Epoch 1674/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 2.8598e-04 - acc: 1.0000\n",
      "Epoch 1675/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0055 - acc: 1.0000\n",
      "Epoch 1676/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0022 - acc: 1.0000\n",
      "Epoch 1677/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0023 - acc: 1.0000\n",
      "Epoch 1678/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 3.7024e-04 - acc: 1.0000\n",
      "Epoch 1679/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0024 - acc: 1.0000\n",
      "Epoch 1680/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0147 - acc: 0.9942\n",
      "Epoch 1681/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 6.2291e-04 - acc: 1.0000\n",
      "Epoch 1682/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 2.6304e-04 - acc: 1.0000\n",
      "Epoch 1683/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 1684/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 5.9295e-04 - acc: 1.0000\n",
      "Epoch 1685/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 8.5502e-04 - acc: 1.0000\n",
      "Epoch 1686/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 2.7467e-04 - acc: 1.0000\n",
      "Epoch 1687/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0033 - acc: 1.0000\n",
      "Epoch 1688/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0057 - acc: 1.0000\n",
      "Epoch 1689/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0012 - acc: 1.0000\n",
      "Epoch 1690/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0013 - acc: 1.0000\n",
      "Epoch 1691/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0021 - acc: 1.0000\n",
      "Epoch 1692/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 3.8882e-04 - acc: 1.0000\n",
      "Epoch 1693/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 4.9456e-04 - acc: 1.0000\n",
      "Epoch 1694/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0072 - acc: 0.9942\n",
      "Epoch 1695/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 5.3197e-04 - acc: 1.0000\n",
      "Epoch 1696/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0106 - acc: 0.9942\n",
      "Epoch 1697/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 4.2513e-04 - acc: 1.0000\n",
      "Epoch 1698/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 6.2631e-04 - acc: 1.0000\n",
      "Epoch 1699/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 1700/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0074 - acc: 0.9942\n",
      "Epoch 1701/2000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "171/171 [==============================] - 0s 50us/step - loss: 2.9063e-04 - acc: 1.0000\n",
      "Epoch 1702/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 8.8858e-04 - acc: 1.0000\n",
      "Epoch 1703/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0021 - acc: 1.0000\n",
      "Epoch 1704/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0173 - acc: 0.9883\n",
      "Epoch 1705/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 2.4480e-04 - acc: 1.0000\n",
      "Epoch 1706/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0045 - acc: 1.0000\n",
      "Epoch 1707/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0044 - acc: 1.0000\n",
      "Epoch 1708/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0162 - acc: 0.9942\n",
      "Epoch 1709/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0012 - acc: 1.0000\n",
      "Epoch 1710/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0055 - acc: 1.0000\n",
      "Epoch 1711/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0012 - acc: 1.0000\n",
      "Epoch 1712/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 2.0428e-04 - acc: 1.0000\n",
      "Epoch 1713/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 4.8339e-04 - acc: 1.0000\n",
      "Epoch 1714/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0012 - acc: 1.0000\n",
      "Epoch 1715/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 2.2949e-04 - acc: 1.0000\n",
      "Epoch 1716/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0012 - acc: 1.0000\n",
      "Epoch 1717/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0022 - acc: 1.0000\n",
      "Epoch 1718/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0021 - acc: 1.0000\n",
      "Epoch 1719/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0017 - acc: 1.0000\n",
      "Epoch 1720/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 1721/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0010 - acc: 1.0000\n",
      "Epoch 1722/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 1.1830e-04 - acc: 1.0000\n",
      "Epoch 1723/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 3.0929e-04 - acc: 1.0000\n",
      "Epoch 1724/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0023 - acc: 1.0000\n",
      "Epoch 1725/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0092 - acc: 0.9942\n",
      "Epoch 1726/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 2.9340e-04 - acc: 1.0000\n",
      "Epoch 1727/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0012 - acc: 1.0000\n",
      "Epoch 1728/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 8.4603e-04 - acc: 1.0000\n",
      "Epoch 1729/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 2.4878e-04 - acc: 1.0000\n",
      "Epoch 1730/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 4.8697e-04 - acc: 1.0000\n",
      "Epoch 1731/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 7.8935e-04 - acc: 1.0000\n",
      "Epoch 1732/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 9.8487e-05 - acc: 1.0000\n",
      "Epoch 1733/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 2.4626e-04 - acc: 1.0000\n",
      "Epoch 1734/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0028 - acc: 1.0000\n",
      "Epoch 1735/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0063 - acc: 0.9942\n",
      "Epoch 1736/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 2.8901e-04 - acc: 1.0000\n",
      "Epoch 1737/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0018 - acc: 1.0000\n",
      "Epoch 1738/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 1739/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 1.5095e-04 - acc: 1.0000\n",
      "Epoch 1740/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 4.1351e-04 - acc: 1.0000\n",
      "Epoch 1741/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 1.1828e-04 - acc: 1.0000\n",
      "Epoch 1742/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0066 - acc: 0.9942\n",
      "Epoch 1743/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0030 - acc: 1.0000\n",
      "Epoch 1744/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0044 - acc: 1.0000\n",
      "Epoch 1745/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 8.8786e-04 - acc: 1.0000\n",
      "Epoch 1746/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 3.0033e-04 - acc: 1.0000\n",
      "Epoch 1747/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0031 - acc: 1.0000\n",
      "Epoch 1748/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 8.3387e-04 - acc: 1.0000\n",
      "Epoch 1749/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0068 - acc: 0.9942\n",
      "Epoch 1750/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0022 - acc: 1.0000\n",
      "Epoch 1751/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 5.1427e-04 - acc: 1.0000\n",
      "Epoch 1752/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0037 - acc: 1.0000\n",
      "Epoch 1753/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 4.3322e-04 - acc: 1.0000\n",
      "Epoch 1754/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 6.2167e-04 - acc: 1.0000\n",
      "Epoch 1755/2000\n",
      "171/171 [==============================] - 0s 39us/step - loss: 2.9486e-04 - acc: 1.0000\n",
      "Epoch 1756/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 2.9241e-04 - acc: 1.0000\n",
      "Epoch 1757/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 7.0775e-04 - acc: 1.0000\n",
      "Epoch 1758/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0047 - acc: 0.9942\n",
      "Epoch 1759/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 2.9967e-04 - acc: 1.0000\n",
      "Epoch 1760/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 9.0102e-04 - acc: 1.0000\n",
      "Epoch 1761/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0145 - acc: 0.9942\n",
      "Epoch 1762/2000\n",
      "171/171 [==============================] - 0s 69us/step - loss: 3.5424e-04 - acc: 1.0000\n",
      "Epoch 1763/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 1764/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 4.5707e-04 - acc: 1.0000\n",
      "Epoch 1765/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0012 - acc: 1.0000\n",
      "Epoch 1766/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 3.8440e-04 - acc: 1.0000\n",
      "Epoch 1767/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 9.4748e-05 - acc: 1.0000\n",
      "Epoch 1768/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 1769/2000\n",
      "171/171 [==============================] - 0s 39us/step - loss: 1.8451e-04 - acc: 1.0000\n",
      "Epoch 1770/2000\n",
      "171/171 [==============================] - 0s 63us/step - loss: 8.2639e-04 - acc: 1.0000\n",
      "Epoch 1771/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 2.0623e-04 - acc: 1.0000\n",
      "Epoch 1772/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 3.0651e-04 - acc: 1.0000\n",
      "Epoch 1773/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 8.4168e-05 - acc: 1.0000\n",
      "Epoch 1774/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 1.8766e-04 - acc: 1.0000\n",
      "Epoch 1775/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 6.5318e-04 - acc: 1.0000\n",
      "Epoch 1776/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 9.0497e-05 - acc: 1.0000\n",
      "Epoch 1777/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 3.8318e-04 - acc: 1.0000\n",
      "Epoch 1778/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 2.2905e-04 - acc: 1.0000\n",
      "Epoch 1779/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 5.9931e-04 - acc: 1.0000\n",
      "Epoch 1780/2000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "171/171 [==============================] - 0s 41us/step - loss: 3.9914e-04 - acc: 1.0000\n",
      "Epoch 1781/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 4.0888e-04 - acc: 1.0000\n",
      "Epoch 1782/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 2.6013e-04 - acc: 1.0000\n",
      "Epoch 1783/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0185 - acc: 0.9942\n",
      "Epoch 1784/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 3.2764e-04 - acc: 1.0000\n",
      "Epoch 1785/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 3.7510e-04 - acc: 1.0000\n",
      "Epoch 1786/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 8.1681e-04 - acc: 1.0000\n",
      "Epoch 1787/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0031 - acc: 1.0000\n",
      "Epoch 1788/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 3.2651e-04 - acc: 1.0000\n",
      "Epoch 1789/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 2.6888e-04 - acc: 1.0000\n",
      "Epoch 1790/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0040 - acc: 1.0000\n",
      "Epoch 1791/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 2.0441e-04 - acc: 1.0000\n",
      "Epoch 1792/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 7.7783e-04 - acc: 1.0000\n",
      "Epoch 1793/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0043 - acc: 1.0000\n",
      "Epoch 1794/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 5.2552e-05 - acc: 1.0000\n",
      "Epoch 1795/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 6.5959e-04 - acc: 1.0000\n",
      "Epoch 1796/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 4.1715e-04 - acc: 1.0000\n",
      "Epoch 1797/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 1.1694e-04 - acc: 1.0000\n",
      "Epoch 1798/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0052 - acc: 0.9942\n",
      "Epoch 1799/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 3.0549e-04 - acc: 1.0000\n",
      "Epoch 1800/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0144 - acc: 0.9942\n",
      "Epoch 1801/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 1.2930e-04 - acc: 1.0000\n",
      "Epoch 1802/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 5.2932e-04 - acc: 1.0000\n",
      "Epoch 1803/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 1804/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 7.5612e-04 - acc: 1.0000\n",
      "Epoch 1805/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0114 - acc: 0.9942\n",
      "Epoch 1806/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0024 - acc: 1.0000\n",
      "Epoch 1807/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 8.4690e-04 - acc: 1.0000\n",
      "Epoch 1808/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0040 - acc: 1.0000\n",
      "Epoch 1809/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0048 - acc: 1.0000\n",
      "Epoch 1810/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 2.6702e-04 - acc: 1.0000\n",
      "Epoch 1811/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0035 - acc: 1.0000\n",
      "Epoch 1812/2000\n",
      "171/171 [==============================] - 0s 60us/step - loss: 0.0051 - acc: 0.9942\n",
      "Epoch 1813/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 8.7810e-05 - acc: 1.0000\n",
      "Epoch 1814/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 5.7801e-04 - acc: 1.0000\n",
      "Epoch 1815/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 9.2267e-04 - acc: 1.0000\n",
      "Epoch 1816/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0041 - acc: 1.0000\n",
      "Epoch 1817/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0017 - acc: 1.0000\n",
      "Epoch 1818/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 3.5269e-04 - acc: 1.0000\n",
      "Epoch 1819/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 1.1273e-04 - acc: 1.0000\n",
      "Epoch 1820/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 4.9261e-04 - acc: 1.0000\n",
      "Epoch 1821/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 2.8857e-04 - acc: 1.0000\n",
      "Epoch 1822/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 4.3370e-04 - acc: 1.0000\n",
      "Epoch 1823/2000\n",
      "171/171 [==============================] - 0s 38us/step - loss: 0.0058 - acc: 0.9942\n",
      "Epoch 1824/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 4.9589e-05 - acc: 1.0000\n",
      "Epoch 1825/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0018 - acc: 1.0000\n",
      "Epoch 1826/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 7.2263e-04 - acc: 1.0000\n",
      "Epoch 1827/2000\n",
      "171/171 [==============================] - 0s 64us/step - loss: 0.0019 - acc: 1.0000\n",
      "Epoch 1828/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 9.0881e-04 - acc: 1.0000\n",
      "Epoch 1829/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 3.4813e-04 - acc: 1.0000\n",
      "Epoch 1830/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0247 - acc: 0.9942\n",
      "Epoch 1831/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0075 - acc: 0.9942\n",
      "Epoch 1832/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0353 - acc: 0.9883\n",
      "Epoch 1833/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0096 - acc: 1.0000\n",
      "Epoch 1834/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0212 - acc: 0.9883\n",
      "Epoch 1835/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 3.2433e-04 - acc: 1.0000\n",
      "Epoch 1836/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0062 - acc: 1.0000\n",
      "Epoch 1837/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0145 - acc: 0.9942\n",
      "Epoch 1838/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0018 - acc: 1.0000\n",
      "Epoch 1839/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0120 - acc: 0.9942\n",
      "Epoch 1840/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0025 - acc: 1.0000\n",
      "Epoch 1841/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0183 - acc: 0.9942\n",
      "Epoch 1842/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0159 - acc: 0.9942\n",
      "Epoch 1843/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0336 - acc: 0.9942\n",
      "Epoch 1844/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0022 - acc: 1.0000\n",
      "Epoch 1845/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 1846/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0012 - acc: 1.0000\n",
      "Epoch 1847/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 1848/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 7.2988e-04 - acc: 1.0000\n",
      "Epoch 1849/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0040 - acc: 1.0000\n",
      "Epoch 1850/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0015 - acc: 1.0000\n",
      "Epoch 1851/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0012 - acc: 1.0000\n",
      "Epoch 1852/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0074 - acc: 0.9942\n",
      "Epoch 1853/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 5.3877e-04 - acc: 1.0000\n",
      "Epoch 1854/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 1855/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0030 - acc: 1.0000\n",
      "Epoch 1856/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 5.0335e-04 - acc: 1.0000\n",
      "Epoch 1857/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0019 - acc: 1.0000\n",
      "Epoch 1858/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0057 - acc: 1.0000\n",
      "Epoch 1859/2000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "171/171 [==============================] - 0s 39us/step - loss: 1.6221e-04 - acc: 1.0000\n",
      "Epoch 1860/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 1.9505e-04 - acc: 1.0000\n",
      "Epoch 1861/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 5.5714e-04 - acc: 1.0000\n",
      "Epoch 1862/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 6.6762e-04 - acc: 1.0000\n",
      "Epoch 1863/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 1864/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 6.2752e-04 - acc: 1.0000\n",
      "Epoch 1865/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 1.0035e-04 - acc: 1.0000\n",
      "Epoch 1866/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0019 - acc: 1.0000\n",
      "Epoch 1867/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 1.9108e-04 - acc: 1.0000\n",
      "Epoch 1868/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 6.8375e-04 - acc: 1.0000\n",
      "Epoch 1869/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 0.0025 - acc: 1.0000\n",
      "Epoch 1870/2000\n",
      "171/171 [==============================] - 0s 39us/step - loss: 5.5105e-04 - acc: 1.0000\n",
      "Epoch 1871/2000\n",
      "171/171 [==============================] - 0s 65us/step - loss: 6.9342e-04 - acc: 1.0000\n",
      "Epoch 1872/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0058 - acc: 0.9942\n",
      "Epoch 1873/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 2.3087e-04 - acc: 1.0000\n",
      "Epoch 1874/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0317 - acc: 0.9883\n",
      "Epoch 1875/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0038 - acc: 1.0000\n",
      "Epoch 1876/2000\n",
      "171/171 [==============================] - 0s 39us/step - loss: 0.0024 - acc: 1.0000\n",
      "Epoch 1877/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0053 - acc: 0.9942\n",
      "Epoch 1878/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 8.1998e-04 - acc: 1.0000\n",
      "Epoch 1879/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 1.9412e-04 - acc: 1.0000\n",
      "Epoch 1880/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 6.2695e-04 - acc: 1.0000\n",
      "Epoch 1881/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 8.2577e-04 - acc: 1.0000\n",
      "Epoch 1882/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0135 - acc: 0.9942\n",
      "Epoch 1883/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0327 - acc: 0.9825\n",
      "Epoch 1884/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0046 - acc: 1.0000\n",
      "Epoch 1885/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0134 - acc: 0.9883\n",
      "Epoch 1886/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0191 - acc: 0.9883\n",
      "Epoch 1887/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0625 - acc: 0.9825\n",
      "Epoch 1888/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0079 - acc: 0.9942\n",
      "Epoch 1889/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0796 - acc: 0.9766\n",
      "Epoch 1890/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0356 - acc: 0.9883\n",
      "Epoch 1891/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0319 - acc: 0.9825\n",
      "Epoch 1892/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0417 - acc: 0.9883\n",
      "Epoch 1893/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0778 - acc: 0.9825\n",
      "Epoch 1894/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0407 - acc: 0.9825\n",
      "Epoch 1895/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.1006 - acc: 0.9766\n",
      "Epoch 1896/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0032 - acc: 1.0000\n",
      "Epoch 1897/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0133 - acc: 0.9942\n",
      "Epoch 1898/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0086 - acc: 0.9942\n",
      "Epoch 1899/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0242 - acc: 0.9825\n",
      "Epoch 1900/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 0.0023 - acc: 1.0000\n",
      "Epoch 1901/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0052 - acc: 1.0000\n",
      "Epoch 1902/2000\n",
      "171/171 [==============================] - 0s 61us/step - loss: 0.0188 - acc: 0.9883\n",
      "Epoch 1903/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0167 - acc: 0.9942\n",
      "Epoch 1904/2000\n",
      "171/171 [==============================] - 0s 39us/step - loss: 0.0152 - acc: 0.9942\n",
      "Epoch 1905/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0273 - acc: 0.9942\n",
      "Epoch 1906/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0038 - acc: 1.0000\n",
      "Epoch 1907/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0059 - acc: 1.0000\n",
      "Epoch 1908/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0137 - acc: 0.9942\n",
      "Epoch 1909/2000\n",
      "171/171 [==============================] - 0s 58us/step - loss: 0.0086 - acc: 1.0000\n",
      "Epoch 1910/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 1911/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 1912/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0162 - acc: 0.9942\n",
      "Epoch 1913/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0137 - acc: 0.9883\n",
      "Epoch 1914/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0189 - acc: 0.9942\n",
      "Epoch 1915/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0273 - acc: 0.9942\n",
      "Epoch 1916/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0061 - acc: 0.9942\n",
      "Epoch 1917/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0256 - acc: 0.9883\n",
      "Epoch 1918/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 0.0053 - acc: 1.0000\n",
      "Epoch 1919/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0027 - acc: 1.0000\n",
      "Epoch 1920/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0039 - acc: 1.0000\n",
      "Epoch 1921/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0020 - acc: 1.0000\n",
      "Epoch 1922/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0064 - acc: 1.0000\n",
      "Epoch 1923/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0066 - acc: 0.9942\n",
      "Epoch 1924/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 7.5589e-04 - acc: 1.0000\n",
      "Epoch 1925/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0121 - acc: 0.9942\n",
      "Epoch 1926/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0064 - acc: 1.0000\n",
      "Epoch 1927/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0021 - acc: 1.0000\n",
      "Epoch 1928/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0124 - acc: 0.9942\n",
      "Epoch 1929/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0028 - acc: 1.0000\n",
      "Epoch 1930/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0080 - acc: 1.0000\n",
      "Epoch 1931/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0038 - acc: 1.0000\n",
      "Epoch 1932/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0028 - acc: 1.0000\n",
      "Epoch 1933/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0084 - acc: 0.9942\n",
      "Epoch 1934/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0195 - acc: 0.9942\n",
      "Epoch 1935/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 1936/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0479 - acc: 0.9883\n",
      "Epoch 1937/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0083 - acc: 1.0000\n",
      "Epoch 1938/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0125 - acc: 0.9883\n",
      "Epoch 1939/2000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "171/171 [==============================] - 0s 49us/step - loss: 0.0078 - acc: 0.9942\n",
      "Epoch 1940/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0069 - acc: 1.0000\n",
      "Epoch 1941/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 1942/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0129 - acc: 0.9942\n",
      "Epoch 1943/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0028 - acc: 1.0000\n",
      "Epoch 1944/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0294 - acc: 0.9883\n",
      "Epoch 1945/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0282 - acc: 0.9883\n",
      "Epoch 1946/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0026 - acc: 1.0000\n",
      "Epoch 1947/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0268 - acc: 0.9942\n",
      "Epoch 1948/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0540 - acc: 0.9825\n",
      "Epoch 1949/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0152 - acc: 0.9942\n",
      "Epoch 1950/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0112 - acc: 0.9942\n",
      "Epoch 1951/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 4.7832e-04 - acc: 1.0000\n",
      "Epoch 1952/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0023 - acc: 1.0000\n",
      "Epoch 1953/2000\n",
      "171/171 [==============================] - 0s 79us/step - loss: 0.0023 - acc: 1.0000\n",
      "Epoch 1954/2000\n",
      "171/171 [==============================] - 0s 63us/step - loss: 0.0026 - acc: 1.0000\n",
      "Epoch 1955/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0021 - acc: 1.0000\n",
      "Epoch 1956/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0023 - acc: 1.0000\n",
      "Epoch 1957/2000\n",
      "171/171 [==============================] - 0s 54us/step - loss: 0.0192 - acc: 0.9942\n",
      "Epoch 1958/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 7.0861e-04 - acc: 1.0000\n",
      "Epoch 1959/2000\n",
      "171/171 [==============================] - 0s 55us/step - loss: 0.0039 - acc: 1.0000\n",
      "Epoch 1960/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0074 - acc: 0.9942\n",
      "Epoch 1961/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0150 - acc: 0.9942\n",
      "Epoch 1962/2000\n",
      "171/171 [==============================] - 0s 38us/step - loss: 9.8752e-04 - acc: 1.0000\n",
      "Epoch 1963/2000\n",
      "171/171 [==============================] - 0s 59us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 1964/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0091 - acc: 0.9942\n",
      "Epoch 1965/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0012 - acc: 1.0000\n",
      "Epoch 1966/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 3.7367e-04 - acc: 1.0000\n",
      "Epoch 1967/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 1.9066e-04 - acc: 1.0000\n",
      "Epoch 1968/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0027 - acc: 1.0000\n",
      "Epoch 1969/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 0.0058 - acc: 0.9942\n",
      "Epoch 1970/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0181 - acc: 0.9942\n",
      "Epoch 1971/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0197 - acc: 0.9942\n",
      "Epoch 1972/2000\n",
      "171/171 [==============================] - 0s 46us/step - loss: 0.0022 - acc: 1.0000\n",
      "Epoch 1973/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0015 - acc: 1.0000\n",
      "Epoch 1974/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0016 - acc: 1.0000\n",
      "Epoch 1975/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 1976/2000\n",
      "171/171 [==============================] - 0s 49us/step - loss: 0.0065 - acc: 0.9942\n",
      "Epoch 1977/2000\n",
      "171/171 [==============================] - 0s 53us/step - loss: 0.0201 - acc: 0.9942\n",
      "Epoch 1978/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0076 - acc: 0.9942\n",
      "Epoch 1979/2000\n",
      "171/171 [==============================] - 0s 41us/step - loss: 0.0017 - acc: 1.0000\n",
      "Epoch 1980/2000\n",
      "171/171 [==============================] - 0s 44us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 1981/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0025 - acc: 1.0000\n",
      "Epoch 1982/2000\n",
      "171/171 [==============================] - 0s 69us/step - loss: 0.0026 - acc: 1.0000\n",
      "Epoch 1983/2000\n",
      "171/171 [==============================] - 0s 60us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 1984/2000\n",
      "171/171 [==============================] - 0s 51us/step - loss: 8.8199e-05 - acc: 1.0000\n",
      "Epoch 1985/2000\n",
      "171/171 [==============================] - 0s 40us/step - loss: 0.0205 - acc: 0.9825\n",
      "Epoch 1986/2000\n",
      "171/171 [==============================] - 0s 57us/step - loss: 6.8073e-04 - acc: 1.0000\n",
      "Epoch 1987/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0063 - acc: 0.9942\n",
      "Epoch 1988/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 8.7034e-04 - acc: 1.0000\n",
      "Epoch 1989/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 0.0042 - acc: 1.0000\n",
      "Epoch 1990/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 1.1740e-04 - acc: 1.0000\n",
      "Epoch 1991/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0011 - acc: 1.0000\n",
      "Epoch 1992/2000\n",
      "171/171 [==============================] - 0s 45us/step - loss: 0.0103 - acc: 0.9942\n",
      "Epoch 1993/2000\n",
      "171/171 [==============================] - 0s 56us/step - loss: 7.7267e-04 - acc: 1.0000\n",
      "Epoch 1994/2000\n",
      "171/171 [==============================] - 0s 47us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 1995/2000\n",
      "171/171 [==============================] - 0s 48us/step - loss: 3.1124e-04 - acc: 1.0000\n",
      "Epoch 1996/2000\n",
      "171/171 [==============================] - 0s 42us/step - loss: 0.0111 - acc: 0.9942\n",
      "Epoch 1997/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0014 - acc: 1.0000\n",
      "Epoch 1998/2000\n",
      "171/171 [==============================] - 0s 43us/step - loss: 0.0015 - acc: 1.0000\n",
      "Epoch 1999/2000\n",
      "171/171 [==============================] - 0s 50us/step - loss: 0.0019 - acc: 1.0000\n",
      "Epoch 2000/2000\n",
      "171/171 [==============================] - 0s 52us/step - loss: 2.3360e-04 - acc: 1.0000\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7fde65312f28>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "opt = SGD(lr=0.01)\n",
    "model = Sequential()\n",
    "model.add(Dense(128, activation='relu', input_shape=(50,)))\n",
    "model.add(Dropout(0.5))\n",
    "model.add(Dense(128, activation='relu'))\n",
    "model.add(Dropout(0.5))\n",
    "model.add(Dense(y.shape[1], activation='softmax'))\n",
    "model.compile(optimizer='Adam',\n",
    "              loss='categorical_crossentropy',\n",
    "              metrics=['accuracy'])\n",
    "model.fit(X1, y1, epochs=2000,batch_size=25)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1,\n",
       "       0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.predict_classes(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0., 0., 1.],\n",
       "       [0., 0., 1.],\n",
       "       [0., 0., 1.],\n",
       "       [0., 1., 0.],\n",
       "       [0., 0., 1.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [0., 1., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [0., 1., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [0., 0., 1.],\n",
       "       [0., 1., 0.],\n",
       "       [0., 0., 1.],\n",
       "       [0., 0., 1.],\n",
       "       [0., 0., 1.],\n",
       "       [0., 1., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [0., 0., 1.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [0., 1., 0.],\n",
       "       [0., 1., 0.],\n",
       "       [0., 1., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [0., 1., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [0., 1., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [0., 0., 1.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [0., 1., 0.],\n",
       "       [0., 0., 1.],\n",
       "       [1., 0., 0.],\n",
       "       [0., 1., 0.],\n",
       "       [0., 0., 1.],\n",
       "       [0., 0., 1.],\n",
       "       [1., 0., 0.],\n",
       "       [0., 0., 1.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [0., 1., 0.],\n",
       "       [0., 0., 1.],\n",
       "       [0., 0., 1.],\n",
       "       [0., 1., 0.]], dtype=float32)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# It Works! Let's try it out and use our original Data as Test\n",
    "\n",
    "Finally, after giving it a once over, it looks like the data mostly matches. Let's ensure it's a perfect fit by using keras' `evaluate` to run a test."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.test_on_batch(X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Hooray! Let's save it now!\n",
    "\n",
    "We use keras' great built in function, `model.save` to save it to disk. \n",
    "\n",
    "In our program for inference, we'll use `keras.models.load_model(filepath)` to load it for deteting dab, tpose, or other."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.save('data/dab-tpose-other.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "import keras\n",
    "modello = keras.models.load_model('data/dab-tpose-other.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 1, 1, 1, 1, 1])"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dabDataset = np.load('data/test-dabs.npy')\n",
    "dabDataset[:,:,0] = dabDataset[:,:,0] / 720 # I think the dimensions are 1280 x 720 ?\n",
    "dabDataset[:,:,1] = dabDataset[:,:,1] / 1280  # let's see?\n",
    "dabDataset = dabDataset[:,:,1:]\n",
    "dabDataset = dabDataset.reshape(len(dabDataset), 50)\n",
    "modello.predict_classes(dabDataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
